diff options
| author | Devtools Arcadia <arcadia-devtools@yandex-team.ru> | 2022-02-07 18:08:42 +0300 |
|---|---|---|
| committer | Devtools Arcadia <arcadia-devtools@mous.vla.yp-c.yandex.net> | 2022-02-07 18:08:42 +0300 |
| commit | 1110808a9d39d4b808aef724c861a2e1a38d2a69 (patch) | |
| tree | e26c9fed0de5d9873cce7e00bc214573dc2195b7 /contrib/tools/ragel6 | |
| download | ydb-1110808a9d39d4b808aef724c861a2e1a38d2a69.tar.gz | |
intermediate changes
ref:cde9a383711a11544ce7e107a78147fb96cc4029
Diffstat (limited to 'contrib/tools/ragel6')
126 files changed, 68882 insertions, 0 deletions
diff --git a/contrib/tools/ragel6/AUTHORS b/contrib/tools/ragel6/AUTHORS new file mode 100644 index 0000000000..7ca82d06ab --- /dev/null +++ b/contrib/tools/ragel6/AUTHORS @@ -0,0 +1 @@ +See CREDITS. diff --git a/contrib/tools/ragel6/COPYING b/contrib/tools/ragel6/COPYING new file mode 100644 index 0000000000..ec0507be1e --- /dev/null +++ b/contrib/tools/ragel6/COPYING @@ -0,0 +1,340 @@ + GNU GENERAL PUBLIC LICENSE + Version 2, June 1991 + + Copyright (C) 1989, 1991 Free Software Foundation, Inc. + 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The licenses for most software are designed to take away your +freedom to share and change it. By contrast, the GNU General Public +License is intended to guarantee your freedom to share and change free +software--to make sure the software is free for all its users. This +General Public License applies to most of the Free Software +Foundation's software and to any other program whose authors commit to +using it. (Some other Free Software Foundation software is covered by +the GNU Library General Public License instead.) You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +this service if you wish), that you receive source code or can get it +if you want it, that you can change the software or use pieces of it +in new free programs; and that you know you can do these things. + + To protect your rights, we need to make restrictions that forbid +anyone to deny you these rights or to ask you to surrender the rights. +These restrictions translate to certain responsibilities for you if you +distribute copies of the software, or if you modify it. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must give the recipients all the rights that +you have. You must make sure that they, too, receive or can get the +source code. And you must show them these terms so they know their +rights. + + We protect your rights with two steps: (1) copyright the software, and +(2) offer you this license which gives you legal permission to copy, +distribute and/or modify the software. + + Also, for each author's protection and ours, we want to make certain +that everyone understands that there is no warranty for this free +software. If the software is modified by someone else and passed on, we +want its recipients to know that what they have is not the original, so +that any problems introduced by others will not reflect on the original +authors' reputations. + + Finally, any free program is threatened constantly by software +patents. We wish to avoid the danger that redistributors of a free +program will individually obtain patent licenses, in effect making the +program proprietary. To prevent this, we have made it clear that any +patent must be licensed for everyone's free use or not licensed at all. + + The precise terms and conditions for copying, distribution and +modification follow. + + GNU GENERAL PUBLIC LICENSE + TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION + + 0. This License applies to any program or other work which contains +a notice placed by the copyright holder saying it may be distributed +under the terms of this General Public License. The "Program", below, +refers to any such program or work, and a "work based on the Program" +means either the Program or any derivative work under copyright law: +that is to say, a work containing the Program or a portion of it, +either verbatim or with modifications and/or translated into another +language. (Hereinafter, translation is included without limitation in +the term "modification".) Each licensee is addressed as "you". + +Activities other than copying, distribution and modification are not +covered by this License; they are outside its scope. The act of +running the Program is not restricted, and the output from the Program +is covered only if its contents constitute a work based on the +Program (independent of having been made by running the Program). +Whether that is true depends on what the Program does. + + 1. You may copy and distribute verbatim copies of the Program's +source code as you receive it, in any medium, provided that you +conspicuously and appropriately publish on each copy an appropriate +copyright notice and disclaimer of warranty; keep intact all the +notices that refer to this License and to the absence of any warranty; +and give any other recipients of the Program a copy of this License +along with the Program. + +You may charge a fee for the physical act of transferring a copy, and +you may at your option offer warranty protection in exchange for a fee. + + 2. You may modify your copy or copies of the Program or any portion +of it, thus forming a work based on the Program, and copy and +distribute such modifications or work under the terms of Section 1 +above, provided that you also meet all of these conditions: + + a) You must cause the modified files to carry prominent notices + stating that you changed the files and the date of any change. + + b) You must cause any work that you distribute or publish, that in + whole or in part contains or is derived from the Program or any + part thereof, to be licensed as a whole at no charge to all third + parties under the terms of this License. + + c) If the modified program normally reads commands interactively + when run, you must cause it, when started running for such + interactive use in the most ordinary way, to print or display an + announcement including an appropriate copyright notice and a + notice that there is no warranty (or else, saying that you provide + a warranty) and that users may redistribute the program under + these conditions, and telling the user how to view a copy of this + License. (Exception: if the Program itself is interactive but + does not normally print such an announcement, your work based on + the Program is not required to print an announcement.) + +These requirements apply to the modified work as a whole. If +identifiable sections of that work are not derived from the Program, +and can be reasonably considered independent and separate works in +themselves, then this License, and its terms, do not apply to those +sections when you distribute them as separate works. But when you +distribute the same sections as part of a whole which is a work based +on the Program, the distribution of the whole must be on the terms of +this License, whose permissions for other licensees extend to the +entire whole, and thus to each and every part regardless of who wrote it. + +Thus, it is not the intent of this section to claim rights or contest +your rights to work written entirely by you; rather, the intent is to +exercise the right to control the distribution of derivative or +collective works based on the Program. + +In addition, mere aggregation of another work not based on the Program +with the Program (or with a work based on the Program) on a volume of +a storage or distribution medium does not bring the other work under +the scope of this License. + + 3. You may copy and distribute the Program (or a work based on it, +under Section 2) in object code or executable form under the terms of +Sections 1 and 2 above provided that you also do one of the following: + + a) Accompany it with the complete corresponding machine-readable + source code, which must be distributed under the terms of Sections + 1 and 2 above on a medium customarily used for software interchange; or, + + b) Accompany it with a written offer, valid for at least three + years, to give any third party, for a charge no more than your + cost of physically performing source distribution, a complete + machine-readable copy of the corresponding source code, to be + distributed under the terms of Sections 1 and 2 above on a medium + customarily used for software interchange; or, + + c) Accompany it with the information you received as to the offer + to distribute corresponding source code. (This alternative is + allowed only for noncommercial distribution and only if you + received the program in object code or executable form with such + an offer, in accord with Subsection b above.) + +The source code for a work means the preferred form of the work for +making modifications to it. For an executable work, complete source +code means all the source code for all modules it contains, plus any +associated interface definition files, plus the scripts used to +control compilation and installation of the executable. However, as a +special exception, the source code distributed need not include +anything that is normally distributed (in either source or binary +form) with the major components (compiler, kernel, and so on) of the +operating system on which the executable runs, unless that component +itself accompanies the executable. + +If distribution of executable or object code is made by offering +access to copy from a designated place, then offering equivalent +access to copy the source code from the same place counts as +distribution of the source code, even though third parties are not +compelled to copy the source along with the object code. + + 4. You may not copy, modify, sublicense, or distribute the Program +except as expressly provided under this License. Any attempt +otherwise to copy, modify, sublicense or distribute the Program is +void, and will automatically terminate your rights under this License. +However, parties who have received copies, or rights, from you under +this License will not have their licenses terminated so long as such +parties remain in full compliance. + + 5. You are not required to accept this License, since you have not +signed it. However, nothing else grants you permission to modify or +distribute the Program or its derivative works. These actions are +prohibited by law if you do not accept this License. Therefore, by +modifying or distributing the Program (or any work based on the +Program), you indicate your acceptance of this License to do so, and +all its terms and conditions for copying, distributing or modifying +the Program or works based on it. + + 6. Each time you redistribute the Program (or any work based on the +Program), the recipient automatically receives a license from the +original licensor to copy, distribute or modify the Program subject to +these terms and conditions. You may not impose any further +restrictions on the recipients' exercise of the rights granted herein. +You are not responsible for enforcing compliance by third parties to +this License. + + 7. If, as a consequence of a court judgment or allegation of patent +infringement or for any other reason (not limited to patent issues), +conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot +distribute so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you +may not distribute the Program at all. For example, if a patent +license would not permit royalty-free redistribution of the Program by +all those who receive copies directly or indirectly through you, then +the only way you could satisfy both it and this License would be to +refrain entirely from distribution of the Program. + +If any portion of this section is held invalid or unenforceable under +any particular circumstance, the balance of the section is intended to +apply and the section as a whole is intended to apply in other +circumstances. + +It is not the purpose of this section to induce you to infringe any +patents or other property right claims or to contest validity of any +such claims; this section has the sole purpose of protecting the +integrity of the free software distribution system, which is +implemented by public license practices. Many people have made +generous contributions to the wide range of software distributed +through that system in reliance on consistent application of that +system; it is up to the author/donor to decide if he or she is willing +to distribute software through any other system and a licensee cannot +impose that choice. + +This section is intended to make thoroughly clear what is believed to +be a consequence of the rest of this License. + + 8. If the distribution and/or use of the Program is restricted in +certain countries either by patents or by copyrighted interfaces, the +original copyright holder who places the Program under this License +may add an explicit geographical distribution limitation excluding +those countries, so that distribution is permitted only in or among +countries not thus excluded. In such case, this License incorporates +the limitation as if written in the body of this License. + + 9. The Free Software Foundation may publish revised and/or new versions +of the General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + +Each version is given a distinguishing version number. If the Program +specifies a version number of this License which applies to it and "any +later version", you have the option of following the terms and conditions +either of that version or of any later version published by the Free +Software Foundation. If the Program does not specify a version number of +this License, you may choose any version ever published by the Free Software +Foundation. + + 10. If you wish to incorporate parts of the Program into other free +programs whose distribution conditions are different, write to the author +to ask for permission. For software which is copyrighted by the Free +Software Foundation, write to the Free Software Foundation; we sometimes +make exceptions for this. Our decision will be guided by the two goals +of preserving the free status of all derivatives of our free software and +of promoting the sharing and reuse of software generally. + + NO WARRANTY + + 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY +FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN +OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES +PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED +OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF +MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS +TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE +PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, +REPAIR OR CORRECTION. + + 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR +REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, +INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING +OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED +TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY +YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER +PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE +POSSIBILITY OF SUCH DAMAGES. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +convey the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + <one line to give the program's name and a brief idea of what it does.> + Copyright (C) <year> <name of author> + + This program is free software; you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation; either version 2 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program; if not, write to the Free Software + Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + + +Also add information on how to contact you by electronic and paper mail. + +If the program is interactive, make it output a short notice like this +when it starts in an interactive mode: + + Gnomovision version 69, Copyright (C) year name of author + Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, the commands you use may +be called something other than `show w' and `show c'; they could even be +mouse-clicks or menu items--whatever suits your program. + +You should also get your employer (if you work as a programmer) or your +school, if any, to sign a "copyright disclaimer" for the program, if +necessary. Here is a sample; alter the names: + + Yoyodyne, Inc., hereby disclaims all copyright interest in the program + `Gnomovision' (which makes passes at compilers) written by James Hacker. + + <signature of Ty Coon>, 1 April 1989 + Ty Coon, President of Vice + +This General Public License does not permit incorporating your program into +proprietary programs. If your program is a subroutine library, you may +consider it more useful to permit linking proprietary applications with the +library. If this is what you want to do, use the GNU Library General +Public License instead of this License. diff --git a/contrib/tools/ragel6/CREDITS b/contrib/tools/ragel6/CREDITS new file mode 100644 index 0000000000..e91a677d63 --- /dev/null +++ b/contrib/tools/ragel6/CREDITS @@ -0,0 +1,50 @@ + + Ragel State Machine Compiler -- CREDITS + ======================================= + + +* Ragel was designed and written by Adrian Thurston <thurston@complang.org>. + +Many others have helped out along the way. My apolgies to anyone who has been +missed. + +* Many thanks to Arbor Networks for supporting development of Ragel. + +* Objective-C output and valuable feedback contributed by Erich Ocean. + +* D output and many great ideas contributed by Alan West. + +* Conditionals inspired by David Helder. + +* Java code generation contributions, bug reports, fixes, test cases + and suggestions from Colin Fleming. + +* Useful discussions and bug reports due to Carlos Antunes. + +* Ruby code generation contributed by Victor Hugo Borja. + +* C# code generation contributed by Daniel Tang. + +* Go code generation contributed by Justine Tunney. Significantly expanded by + Anton Ageev + +* D2 patch from Johannes Pfau. + +* OCaml patch from ygrek. + +* Feedback, Packaging, and Fixes provided by: + + Bob Tennent, Robert Lemmen, Tobias Jahn, Cris Bailiff, Buddy Betts, + Scott Dixon, Steven Handerson, Michael Somos, Bob Paddock, Istvan Buki, + David Drai, Matthias Rahlf, Zinx Verituse, Markus W. Weissmann, + Marc Liyanage, Erich Ocean, Alan West, Steven Kibbler, Laurent Boulard, + Jon Oberheide, David Helder, Lexington Luthor, Jason Jobe, Colin Fleming, + Carlos Antunes, Steve Horne, Matt Mower, Josef Goettgens, Zed Shaw, + Marcus Rueckert, Jeremy Hinegardner, Aaron Campbell, Josh Purinton, + Judson Lester, Barry Arthur, Tim Potter, Ryan Phelps, David Waite, + Kenny MacDermid, MenTaLguY, Manoj Rajagopalan, Tim Chklovski, + Mikkel Fahnøe Jørgensen, Andrei Polushin, Evan Phoenix, David Balmain, + Ross Thomas, Mitchell Foral, John D. Mitchell, Diego 'Flameeyes' Pettenò, + Jose Quinteiro, William Morgan, _why, Iñaki Baz Castillo, Attila Sztupák, + Ismael Luceno, Josh Stern, Denis Naumov, Anton Ageev, Kamil Klimkiewicz, + Hesham Wahba, Phil Carmody diff --git a/contrib/tools/ragel6/ChangeLog b/contrib/tools/ragel6/ChangeLog new file mode 100644 index 0000000000..d55394e986 --- /dev/null +++ b/contrib/tools/ragel6/ChangeLog @@ -0,0 +1,1680 @@ +Ragel 6.10 - Mar 24, 2017 +========================= + -C codegen: test P vs PE in goto/call/ret statements in EOF actions, just + before re-entering. If at the end of the input block then the EOF check is + jumped to. This change prevents overrunning the buffer if control flow is + issued in an EOF action without fixing the input pointer first. If a program + properly issues an fhold before the control flow the program won't be + affected. + -Updated action label generation. The previous set of conditions for + generating the label didn't cover actions coming from the eofAction pointer + (eof trans covered since it points into the set of transitions). + -Use separate signed/unsigned values for host type min/max. Using separate + values avoids the need to type cast before the data goes into FsmCtx structs. + Keep it in native types until it is used. + -Optionally do not generate entry point variables. Adds noentry write option + for data. + -Various warning elimination and build updates. + +Ragel 6.9 - Oct 13, 2014 +======================== + -updated command-line synopsis + -ocaml: fix missing semicolon + -ocaml: support -G1 + -ocaml: choose a unique name for type state + -ruby: reduce the amount of calls to GET_WIDE_KEY() + -union test case: warning fix + -omit line directives around expression-oriented write statements + -use AS_IF and test command to check if the DIST file is present + -added missing std:: using + -go: added '//line' directive support + +Ragel 6.8 - Feb 11, 2013 +======================== + + -The -G2 code generator for Go1 was rewritten. Table, flat and switch-based + code generators were added. (Anton Ageev) + -The CXXFLAGS variable is not longer set in the configure script. + +Ragel 6.7 - May 22, 2011 +======================== + -The C vim file now supports L,l on the end of literal numbers, other syntax + highlighting fixes. + -Added an explicit cast when modifying _trans to eliminate compiler warnings + on Windows/VC++ + -Fix for ruby 1.9 compatibility. + -Fix for include directories (-I option) on Windows/VC++ + -Can now rename data variable in C# code generator. + -Fix for non-char alphtype in C# code generator. + -Fix for signedness of wide char in C code generator. when comparing the wide + type against a literal we need to pick the right signedness for the literal. + -Fixed arithmetic overflow in generated C# code. The C# _acts and _nacts vars + should not be typed based on the size of the array elements they point to. + Fixes a bug reported by Attila Sztupák. + -Made the -L option work for Ruby. + -Enhanced ragel.m4 (from Diego). + -Applied GO patch from Justine Tunney. + -Applied D2 patch from Johannes Pfau. + -Applied Ocaml patch from ygrek. + +Ragel 6.6 - Dec 2, 2009 +======================= + -Applied a number of patches from Diego Elio 'Flameeyes' Pettenò. Should not + be modifying the program's arguments. Problem raised by const correctness in + gcc 4.4. Other const-correctness and include fixes provided. + -Fixed improper allocation of checks in makeIncludePathChecks. + -Fixed segfault when there are no machine instantiations. + -Fixed wrong line directives. Line directives need to use the fileName stored + in the InputLoc stuctures from the parse trees, not the root source file, + otherwise actions in included files will have the wrong source file names + associated with the text. + -Made a number of build system improvements. We locate the DIST file using + $srcdir and source it. It contains settings for build_parsers and + build_manual. This allows the user of a dist to enable only one. + -Added missing files to doc/Makefile.am and examples/Makefile.am. + -Added checks for pdflatex and fig2dev is build_manual is on. + -Use automake --foreign so we don't need to have INSTALL and NEWS present. + -Ragel VIM syntax files should be specialized by host language. Updated the + VIM syntax files. + -Added examples to the dist. Added unicode2ragel.rb to EXTRA_DIST in contrib. + -Moved unicode2ragel.rb to the contrib directory. + +Ragel 6.5 - May 18, 2009 +======================== + -Fixed a bug in graphviz generation. Ragel crashed when using -V and -M and + the specified machine referenced another machine that wasn't included in the + build. + -The name "CS" is in use on OpenSolaris, changed to vCS to ease compiling + Ragel there. + -Converted to automake. + -REALLY fixed a bug that was intended to be fixed in 6.4: + Fixed a problem reading hex numbers that have the high bit set when the + alphabet is signed and we are on 64 bit. This was reported by _why. The + fix was provided by Wialliam Morgan. The literal 0xffffffff was used for + a fully set long when -1L should be used instead. + A null patch (whitespace changes) must have gotten checked after I was + testing with and without the critical one-line patch and I forgot to enable + make sure it was enabled in the final checkin version. + +Ragel 6.4 - Mar 22, 2009 +======================== + -Moved back to a single executable. The old intermediate format can still be + generated using the -x option. Ragel was split into frontend and backend + programs in version 5.0. This was done to encourage interoperability with + other tools. Since then, ragel has been made to work with qfsm, with ragel + producing the intermediate format and qfsm consuming it. However, there has + been no use of Ragel as a consumer of state machine data, with Ragel used as + a code generator for DFAs. This is not surprising given that much of the + complexity of Ragel is in the frontend, where the regular language to DFA + compilation happens. Since the full benefits of the split have not + materialized, and the split increases the complexity for users, Ragel has + been made once again into a single executable. + -Applied a fix to the documentation Makefile from John D. Mitchell. + -Use CXXFLAGS instead of CFLAGS for C++ compiling. Patch from Diego + 'Flameeyes' Pettenò. + -Added support for DESTDIR variable. Patch from Diego 'Flameeyes' Pettenò. + -Added a script called unicode2ragel.rb for generating unicode machines to + the examples directory. From Rakan El-Khalil. + -Fixed a copy-paste error in the documentation that was reported by Jose + Quinteiro. + -Added three new write commands: + write start; + write first_final; + write error; + These generate a reference to the start, first final and error state. When + there are many different machine specifications in one file it is easy to + get the prefix for these wrong (especially when you do a lot of copy-pasting + of boilerplate). The problem can be avoided by using write commands. + -Fixed a problem reading hex numbers that have the high bit set when the + alphabet is signed and we are on 64 bit. This was reported by _why. The fix + was provided by Wialliam Morgan. The literal 0xffffffff was used for a fully + set long when -1L should be used instead. + +Ragel 6.3 - Aug 29, 2008 +======================== + -Fixed an assertion that is too strong. In the condition code we need to copy + transitions that have non-empty lmActionTable arrays so we don't assert + emptiness in the constructor. Lift out the assertion and copy the array in + the constructor. + -Fixed and improved multiple include prevention. We now track the entire + include history of a parser state to prevent duplicates. + -Fixed crash on failed lookup of goto/call/etc target. + +Ragel 6.2 - May 9, 2008 +======================= + -Bug fix: The lm_switch actions needs to set p from tokend when there is no + user action. + -Bug fix: when not using indicies we can't use a transitions's id to identify + the eof transition to take. Instead add the transition to the end of the + transition list and store its position in a new var called pos. The pos var + is then used as the index. + -Bug fix: an fnext followed by an fbreak in -G2 was not working. The fbreak + was not aware that the fnext causes the cs variable to be forced active. In + this case fbreak does not need to save cs because it is already current. + -Bug fix: need to compute the low and high character-space keys from the + condition-trans overlap when computing an expansion. Can't use the range + supplied from the condition overlap since they may not match. An incorrect + machine that accepted 1(!cond1, !cond2) was generated for the following + grammar. This bug was reported by Tim Chklovski. + c = 2 @matched_c; + sc1 = 1..2 when cond1; + sc2 = 1..2 when cond2; + main := sc1 | c | sc2; + -Bug fix: error messages in start label analysis of join operations were + causing assertion failures because location info was not set. Fixed by + adding locations. + -Include and import file searching now searches for the file name given based + on the location of the current file, not ragel's current path. + Additional search locations can be given using the -I option. + -Rubinius code generation was updated to the latest Rubinius. Patch from Evan + Phoenix. + -Switched from strcasecmp to strcmp for testing long arguments. + -Applied a patch from Andrei Polushin for setting the error message format. + --error-format=gnu (default) + --error-fromat=msvc + -Now using the _WIN32 define instead of _WIN32. Other MSVC compilation + improvments from Andrei Polushin. + -Added the hyperref package to the manual. + +Ragel 6.1 - Mar 26, 2008 +======================== + -Scanners now ensure that any leaving actions at the end of a pattern are + executed. They are always executed before the pattern action. + -Added an option -d for turning off the removal of duplicate actions from + actions lists. + -Need to unset the final state status of the start state in kleene star if it + is set. It is possible to crash ragel when the warning is ignored. + -In the dot file generation we need to print any actions that are in + State::eofTrans. These come from scanners only. + -Use @docdir@ for the docdir Makefile variable. + -Check for ar and ranlib in the configure script. + +Ragel 6.0 - Jan 12, 2008 +======================== + -Removed the 'noend' write option from examples/atoi.rl. This example is + referenced a lot as a first example and as such it shouldn't contain a + special purpose write option like 'noend'. + -Introcuded the "eof" variable for indicating the end of file. The p variable + is checked against eof when the processing loop reaches the end of a block. + If p == eof at this time then the EOF actions are executed. The variable is + required only when EOF actions have been emebedded. + -The "write eof" command is no longer needed and was removed. + -Scanners now use EOF actions to generate tokens. This eliminates the need to + flush the last token. + -Restructured the Java driver; a switch statement with fallthrough cases are + now used to emulate gotos. + -Ruby code generation was also restructured. Gotos are elmulated using a + series of if tests. + -Went back to 3.X semantics for >, % and error actions. The > operator also + embeds a leaving action/priority into the start state if it is final. If EOF + happens in a state with a leaving operator then the leaving action is + executed. If EOF happens in a non-final state that has an error action, the + error action is executed. + -The "ragel" program now executes frontend and backend processes separately, + connecting them with a temporary file in the current directory. Without the + -x option the "ragel" program marshals arguments and calls the frontend and + backend. With the -x option the "ragel" program acts as the frontend only. + -Added name finding for executables. If any forward slash is found in argv0 + then it is assumed that the path is explicit and the path to the backend + executable should be derived from that. Whe check that location and also go + up one then inside a directory of the same name in case we are executing + from the source tree. If no forward slash is found it is assumed the file is + being run from the installed location. The PREFIX supplied during + configuration is used. + -On windows GetModuleFileNameEx is used to find out where the the current + process's binary is. That location is searched first. If that fails then we + go up one directory and look for the executable inside a directory of the + same name in case we are executing from the source tree. + -Changed the -l option in rlgen-cd to -L because it is covered in the + frontend. Added a passthrough in the frontend for the backend options. + -Dot file generation can now be invoked using the -V option to ragel. We + now require an input file. If standard in is used then we don't have a file + name on which to base the output. + -Able to build native windows executables using Cygwin+MinGW. + -Patch from David Waite: Large arrays are now created by copying in the data + from smaller arrays using System.arraycopy(). This eliminates the debug data + associated with explicit initialization statements. It is also much easier + on the java compiler which can run out of memory compiling very large + machines. The downside is that it takes slightly longer to initialize static + data at run time. + -The fbreak statement now advances p. + -In the :> :>> and <: operators it was possible for the priority assignment + to be bypassed via the zero length string. In :> this was fixed + automatically with the semantics change to the entering priority operator. + If the start state is final it now embeds a leaving action into it, + preventing persistance through the zero length string. In :>> and <: this + was fixed explicitly. With <: the entering priority operator was used and + with :> a special exception was added. Since it uses the finishing + transition operator it also adds a leaving priority to the start state if it + is final. + -Ranlib is now run on the archives. Patch from Kenny MacDermid. + -The case statement syntax in ruby code generation used a form depreciated in + Ruby 1.9. Updated it. + -Made a number of fixes that eliminate warnings in GCC 4.3. Mostly concern + the now depreciate automatic conversion of string contsants to "char*" type. + Other fixes include adding parenthesis around && within ||. + -The "tokstart" and "tokend" variables were changed to "ts" and "te". + +Ragel 5.25 - Dec 24, 2007 +========================= + -Fixed segfault reported by Ryan Phelps. Affected Java and Ruby code + generation. The dataExpr variable was not initialized. + -Fixed incorrect case label in test/runtests. Caused Objective-C tests to be + ignored. + -Added missing include to common.cpp. + +Ragel 5.24 - Sep 16, 2007 +========================= + -Applied patch from Victor Hugo Borja <vic@rubyforge.org>. This patch + implements -T1 -F0 -F1 and -G0 in the ruby code generator. Goto-driven code + generation is experimental and requires rubinius asm directives (specify + with --rbx option). These code generators pass all the ruby tests. + -If the condition embedding code runs out of available characters in the + keyspace an error message is emitted. + -The first example that appeared in the manual used the special-purpose + 'noend' write option. This caused confusion. Now a basic example appears + first. + -Added two new statements: prepush and postpop. These are code blocks that + are written out during call and return statements. The prepush code is + written immediately before pushing the current state to the state stack + during a call. The postpop code is written immediately after popping the + current state during return. These can be used to implement a dynamically + resizable stack. + +Ragel 5.23 - Jul 24, 2007 +========================= + -Eliminated the use of callcc as an alternative to goto. Instead, the named + breaks implementation used in the Java code generator is imitated using + control flow variables. + -Improved the error message given when there is a write statement but no + machine instantiations and hence no state machine. + -Documentation improvements: updates to "Machine Instantiation", "Write Init" + and "Write Exports" sectons. Added the "Variables Used by Ragel" section. + -Renamed "Entering Actions" to "Starting Actions." + -Other documentation updates. + +Ragel 5.22 - June 14, 2007 +========================== + -Bug fix: need to isolate the start state of a scanner before setting the + to-state and from-state actions which clear and set tokstart. This affected + very simple scanners only. Most scanners have an isolated start state due to + the pattern structure. + -Bug fix: when -S or -M was given the ragel version number was not emitted, + causing the backend to reject the intermediate format. From Tim Potter. + -The p varialbe is now set up at the beginning of a scanner action, rather + than at the end. This leaves scanner actions free to manipulate p and + removes the need for the special holdTE and execTE (TE for tokend) versions + of hold and exec. It also removes the need to set p = tokend-1 immediately + before any control flow. We loose the ability to determine where in the + input stream a scanner action is executed, however this information is of + little use because it is primarily an artifact of the scanner implementation + (sometimes the last char, other times later on). The gains of this change + are consistency and simplicity. + -The "data" variable (used in Java and Ruby code generation only) can now be + overridden using the variable statement. + +Ragel 5.21 - May 9, 2007 +======================== + -Fixed an inconsistency in the value of p following an error. In the C + directly executable code (rlgen-cd -G2) p is left at the character where + the error occurred, which is correct. In all other code generators it was + left at the following character. This was fixed. Now in all code generators + p is left at the character where the error occurred. + -Bug fix: when fhold was used in scanner pattern actions which get executed + on the last character of the pattern (pattern matches which do not require + any lookahead), fhold was modifying p instead of tokend. This was fixed and + the patact.rl test was modified to cover the case. + -Fixed typos in the guide, improved the state action embedding operator + section and added subsections on the variable, import, and export + statements. + -Implemented a better solution than the pri hack for resolving the '-' + ambiguity: force a shortest match of term. + -Fixed bugs in the binary searching for condition keys in both the Ruby and + Java code generation. + -Can now embed the negative sense of a condition. Added a language- + independent test case for this feature and the necessary transformation + support. + -Added new condition embedding syntax: + expr inwhen cond - The transitions into the machine (starting transitions). + expr outwhen cond - The pending transitions out of the machine. + -The argument to the variable statement which affects the name of the current + state variable was changed from "curstate" to "cs" (the default name used + for the current state) + -Implemented the other variables names in the variable statement. Now all + variables (p, pe, cs, top, stack, act, tokstart, tokend) can be renamed. + -Parse errors in the intermediate XML file now cause the backend to exit + immediately rather then forge on. The recovery infrastructure isn't there + and segfaults are likely. + -When no input is given to the backend program, it should not print an error + message, it should just return a non-zero exit status. The assumption is + that the frontend printed an error. + -The version number is now included in the intermediate file. An error is + emitted if there is a mismatch. + -The alphabet type is now communicated from the frontend to the backend using + a one-word internal name instead of an array offset. + -The Ruby host language types had been just copied from Java. Reduced them to + two basic types: char and int, both signed with the usual C sizes. + +Ragel 5.20 - Apr 7, 2007 +======================== + -The cs variable is now always initialized, unless the "nocs" option is given + to the write init command. If there is no main machine, cs is initialized to + the entry point defined by the last machine instantiation. + -A number of fixes were made to the Ruby code generator. + -The frontend now scans ruby comments and regular expressions. + -A transformation for Ruby was added to the language-independent test suite. + The Ruby code generator passes on all the language-independent tests. + -A new Ruby test and two language-independent tests were added. + -Some portability fixes were made (Patches from Josef Goettgens and Aaron + Campbell). + -Fixed a make dependency bug which caused a problem for parallel building + (Patch from Jeremy Hinegardner). + +Ragel 5.19 - Mar 14, 2007 +========================= + -Added an import statement to ragel. This statement takes a literal string as + an argument, interprets it as a file name, then scrapes the file for + sequences of tokens that match the following forms. Tokens inside ragel + sections are ignored. An example is in test/import1.rl + name = number + name = lit_string + "define" name number + "define" name lit_string + -Added an export mechanism which writes defines for single character machines + that have been tagged with the export keyword in their definition. Defines + are used for C, ints for D, Java and Ruby. Examples of the export feature + are in test/export*.rl. + -All machine instantiations are now always generated, even if they are not + referenced. In the backend, entry points for all instantiations are written + out alongside start, error and first final states. + -If the main machine is not present then do not emit an error. Generate the + machine without a start state and do not initialize cs in the write init + code. + -Added an option -l to rlgen-cd which inhibits the writing of #line + directives. + -Added a new syntax for verbose embeddings. This adds parentheses: + $from(action_name); + Verbose embeddings without parentheses can make code difficult to read + because they force a space in the middle of an action embedding. There is a + tendency to associtate spaces with concatenation. Without syntax + highlighting to make it clear that the embedding type is a keyword, the + problem is especially bad. The danger is that a verbose embedding could be + read as an embedding of the keyword representing the empbedding type. With + parentheses, verbose embeddings read much more clearly. + -Conditions now have a forced order when more than one is executed on a + single character. Previously ordering relied on pointers, which caused + results to vary by compiler. Ordering is now done using conditon action + declaration order. This fixes the failure of cond4.rl which occured with + g++ 4.1 and other compiler versions. + -In the port from flex to ragel, the name separator :: in Ragel code was + lost. Added it back. + -In the examples directory switched from rlcodegen to rlgen-cd. Silenced a + warning in statechart.rl. + -In the root makefile the distclean target was fixed. It was calling clean in + the subdirs. In docs, the clean target was not deleting the new manpages for + the rlgen-* programs. Fixed. + -Portability and other fixes from Josef Goettgens were applied. + -The @datadir@ and @mandir@ variables are made use of in doc/Makefile.in for + specifying where documentation should be installed. Patch from Marcus + Rueckert. + +Ragel 5.18 - Feb 13, 2007 +========================= + -There is now a 100% correspondence between state id numbers in the + intermediate XML file, Graphviz dot files and generated code. This was + achieved by moving code which determines if the error state is necessary + into the frontend, and then assigning state numbers before writing out the + intermediate file. + -Backened class structure was reorganized to make it easier to add new code + generators without having to also modify the existing code generators. + -The C and D code generation executable was changed to rlgen-cd. + -The Java code generation was split out into it's own exectuable (rlgen-java) + to allow it to freely diverge from the C/D-based code generation. + -The graphviz dot file generation was also split out to it's own executable + (rlgen-dot). + -The Ruby code generation patch from Victor Hugo Borja was added. This is + highly experimental code and is not yet completely functional. It is in the + executable rlgen-ruby. + -The problem with large state machine machines in Java was fixed. This + problem was discovered by Colin Fleming, who also contributed a patch. + Rather than specify arrays as comma-separated lists of literals, array + initialization is now done in a static function. This is the approach used + by the Java compiler. Unlike the compiler Ragel is careful split large + initilization functions. + -The manual was expanded and reorganized somewhat. + -Eliminated per-example directories in examples/. + -Made some fixes to the pullscan.rl example. + -In the frontend CR characters are now treated as whitespace. + -Updated to the latest aapl. This completely eliminates the shallowCopy + function. With that, a definitive memory leak is fixed. + -Control codes with escape sequences are now printable characters (-p + option). Also, the space character is now printed as SP. + -Fixed the null dereference and consequential segfault which occurred when + trying to create empty machines with [] and // and /a[]b/. + -Fixed the segfault which occured when a machine reference failed. + -Discontinuing ragel.spec. It is more appropriate for this to be written by + package maintenance developers. + +Ragel 5.17 - Jan 28, 2007 +========================= + -The scanners and parsers in both the frontend and backend programs were + completely rewritten using Ragel and Kelbt. + -The '%when condition' syntax was functioning like '$when condition'. This + was fixed. + -In the Vim syntax file fixes to the matching of embedding operators were + made. Also, improvements to the sync patterns were made. + -Added pullscan.rl to the examples directory. It is an example of doing + pull-based scanning. Also, xmlscan.rl in rlcodegen is a pull scanner. + -The introduction chapter of the manual was improved. The manually-drawn + figures for the examples were replaced with graphviz-drawn figures. + +Ragel 5.16 - Nov 20, 2006 +========================= + -Policy change: the fhold and fexec directives did not function correctly in + scanner pattern actions. In this context manipulations of p may be lost or + made invalid. In the previous version of Ragel they were banned because of + this. Instead of banning these directives they have been fixed. The fexec + and fhold directives now manipulate tokend, which is now always used to + update p when the action terminates. + +Ragel 5.15 - Oct 31, 2006 +========================= + -A language independent test harness was introduced. Test cases can be + written using a custom mini-language in the embedded actions. This + mini-language is then translated to C, D and Java when generating the + language-specific test cases. + -Several existing tests have been ported to the language-independent format + and a number of new language-independent test cases have been added. + -The state-based embedding operators which access states that are not the + start state and are not final (the 'middle' states) have changed. They + were: + <@/ eof action into middle states + <@! error action into middle states + <@^ local error action into middle states + <@~ to-state action into middle states + <@* from-state action into middle states + They are now: + <>/ eof action into middle states + <>! error action into middle states + <>^ local error action into middle states + <>~ to-state action into middle states + <>* from-state action into middle states + -The verbose form of embeddings using the <- operator have been removed. + This syntax was difficult to remember. + -A new verbose form of state-based embedding operators have been added. + These are like the symbol versions, except they replace the symbols: + / ! ^ ~ * + with literal keywords: + eof err lerr to from + -The following words have been promoted to keywords: + when eof err lerr to from + -The write statment now gets its own lexical scope in the scanner to ensure + that commands are passed through as is (not affected by keywords). + -Bug fix: in the code generation of fret in scanner actions the adjustment to + p that is needed in some cases (dependent on content of patterns) was not + happening. + -The fhold directive, which decrements p, cannot be permitted in the pattern + action of a scanner item because it will not behave consistently. At the end + of a pattern action p could be decremented, set to a new value or left + alone. This depends on the contents of the scanner's patterns. The user + cannot be expected to predict what will happen to p. + -Conditions in D code require a cast to the widec type when computing widec. + -Like Java, D code also needs if (true) branches for control flow in actions + in order to fool the unreachable code detector. This is now abstracted in + all code generators using the CTRL_FLOW() function. + -The NULL_ITEM value in java code should be -1. This is needed for + maintaining tokstart. + +Ragel 5.14 - Oct 1, 2006 +======================== + -Fixed the check for use of fcall in actions embedded within longest match + items. It was emitting an error if an item's longest-match action had an + fcall, which is allowed. This bug was introduced while fixing a segfault in + version 5.8. + -A new minimization option was added: MinimizeMostOps (-l). This option + minimizes at every operation except on chains of expressions and chains of + terms (eg, union and concat). On these chains it minimizes only at the last + operation. This makes test cases with many states compile faster, without + killing the performance on grammars like strings2.rl. + -The -l minimiziation option was made the default. + -Fixes to Java code: Use of the fc value did not work, now fixed. Static data + is now declared with the final keyword. Patch from Colin Fleming. Conditions + now work when generating Java code. + -The option -p was added to rlcodegen which causes printable characters to be + printed in GraphViz output. Patch from Colin Fleming. + -The "element" keyword no longer exists, removed from vim syntax file. + Updated keyword highlighting. + -The host language selection is now made in the frontend. + -Native host language types are now used when specifying the alphtype. + Previously all languages used the set defined by C, and these were mapped to + the appropriate type in the backend. + +Ragel 5.13 - Sep 7, 2006 +======================== + -Fixed a careless error which broke Java code generation. + +Ragel 5.12 - Sep 7, 2006 +======================== + -The -o flag did not work in combination with -V. This was fixed. + -The split code generation format uses only the required number of digits + when writing out the number in the file name of each part. + -The -T0, -F0 and -G0 codegens should write out the action list iteration + variables only when there are regular, to state or from state actions. The + code gens should not use anyActions(). + -If two states have the same EOF actions, they are written out in the finish + routine as one case. + -The split and in-place goto formats would sometimes generate _out when it is + not needed. This was fixed. + -Improved the basic partitioning in the split code gen. The last partition + would sometimes be empty. This was fixed. + -Use of 'fcall *' was not causing top to be initialized. Fixed. + -Implemented a Java backend, specified with -J. Only the table-based format + is supported. + -Implemented range compression in the frontend. This has no effect on the + generated code, however it reduces the work of the backend and any programs + that read the intermediate format. + +Ragel 5.11 - Aug 10, 2006 +========================= + -Added a variable to the configure.in script which allows the building of + the parsers to be turned off (BUILD_PARSERS). Parser building is off by + default for released versions. + -Removed configure tests for bison defines header file. Use --defines=file + instead. + -Configure script doesn't test for bison, flex and gperf when building of the + parsers is turned off. + -Removed check for YYLTYPE structure from configure script. Since shipped + code will not build parsers by default, we don't need to be as accomodating + of other versions of bison. + -Added a missing include that showed up with g++ 2.95.3. + -Failed configure test for Objective-C compiler is now silent. + +Ragel 5.10 - Jul 31, 2006 +========================= + -Moved the check for error state higher in the table-based processing loop. + -Replaced naive implementations of condition searching with proper ones. In + the table-based formats the searching is also table-based. In the directly + executed formats the searching is also directly executable. + -The minimization process was made aware of conditions. + -A problem with the condition implementation was fixed. Previously we were + taking pointers to transitions and then using them after a call to + outTransCopy, which was a bad idea because they may be changed by the call. + -Added test mailbox3.rl which is based on mailbox2.rl but includes conditions + for restricting header and message body lengths. + -Eliminated the initial one-character backup of p just before resuming + execution. + -Added the -s option to the frontend for printing statistics. This currently + includes just the number of states. + -Sped up the generation of the in-place goto-driven (-G2) code style. + -Implemented a split version of in-place goto-driven code style. This code + generation style is suitable for producing fast implementations of very + large machines. Partitioning is currently naive. In the future a + high-quality partitioning program will be employed. The flag for accessing + this feature is -Pn, where n is the number of partitions. + -Converted mailbox1.rl, strings2.rl and cppscan1.rl tests to support the + split code generation. + -Fixes and updates were made to the runtests script: added -c for compiling + only, changed the -me option to -e, and added support for testing the split + code style. + +Ragel 5.9 - Jul 19, 2006 +======================== + -Fixed a bug in the include system which caused malformed output from the + frontend when the include was made from a multi-line machine spec and the + included file ended in a single line spec (or vice versa). + -Static data is now const. + -Actions which referenced states but were not embedded caused the frontend to + segfault, now fixed. + -Manual now built with pdflatex. + -The manual was reorganized and expanded. Chapter sequence is now: + Introduction, Constructing Machines, Embedding Actions, Controlling + Nondeterminism and Interfacing to the Host program. + +Ragel 5.8 - Jun 17, 2006 +======================== + -The internal representation of the alphabet type has been encapsulated + into a class and all operations on it have been defined as C++ operators. + -The condition implementation now supports range transitions. This allows + conditions to be embedded into arbitrary machines. Conditions are still + exprimental. + -More condition embedding operators were added + 1. Isolate the start state and embed a condition into all transitions + leaving it: + >when cond OR >?cond + 2. Embed a condition into all transitions: + when cond OR $when cond OR $?cond + 3. Embed a condition into pending out transitions: + %when cond OR %?cond + -Improvements were made to the determinization process to support pending out + conditions. + -The Vim sytax file was fixed so that :> doesn't cause the match of a label. + -The test suite was converted to a single-file format which uses less disk + space than the old directory-per-test format. + +Ragel 5.7 - May 14, 2006 +======================== + -Conditions will not be embedded like actions because they involve a + manipulation of the state machine they are specified in. They have therefore + been taken out of the verbose action embedding form (using the <- compound + symbol). A new syntax for specifying conditions has been created: + m = '\n' when {i==4}; + -Fixed a bug which prevented state machine commands like fcurs, fcall, fret, + etc, from being accounted for in from-state actions and to-state actions. + This prevented some necessary support code from being generated. + -Implemented condition testing in remaining code generators. + -Configure script now checks for gperf, which is required for building. + -Added support for case-insensitive literal strings (in addition to regexes). + A case-insensitive string is made by appending an 'i' to the literal, as in + 'cmd'i or "cmd"i. + -Fixed a bug which caused all or expressions inside of all regular + expressions to be case-insensitive. For example /[fo]o bar/ would make the + [fo] part case-insensitive even though no 'i' was given following the + regular expression. + +Ragel 5.6 - Apr 1, 2006 +======================= + -Added a left-guarded concatenation operator. This operator <: is equivalent + to ( expr1 $1 . expr2 >0 ). It is useful if you want to prefix a sequence + with a sequence of a subset of the characters it matches. For example, one + can consume leading whitespace before tokenizing a sequence of whitespace + separated words: ( ' '* <: ( ' '+ | [a-z]+ )** ) + -Removed context embedding code, which has been dead since 5.0. + +Ragel 5.5 - Mar 28, 2006 +======================== + -Implemented a case-insensitive option for regular expressions: /get/i. + -If no input file is given to the ragel program it reads from standard input. + -The label of the start state has been changed from START to IN to save on + required screen space. + -Bug fix: \0 was not working in literal strings, due to a change that reduced + memory usage by concatenating components of literal strings. Token data + length is now passed from the scanner to the paser so that we do not need to + rely on null termination. + +Ragel 5.4 - Mar 12, 2006 +======================== + -Eliminated the default transition from the frontend implementation. This + default transition was a space-saving optimization that at best could reduce + the number of allocated transitions by one half. Unfortunately it + complicated the implementation and this stood in the way of introducing + conditionals. The default transition may be reintroduced in the future. + -Added entry-guarded concatenation. This operator :>, is syntactic sugar + for expr1 $0 . expr >1. This operator terminates the matching of the first + machine when a first character of the second machine is matched. For + example in any* . ';' we never leave the any* machine. If we use any* :> ';' + then the any* machine is terminiated upon matching the semi-colon. + -Added finish-guarded concatenation. This operator :>>, is syntactic sugar + for expr1 $0 . expr @1. This operator is like entry guarded concatenation + except the first machine is terminated when the second machine enters a + final state. This is useful for delaying the guard until a full pattern is + matched. For example as in '/*' any* :>> '*/'. + -Added strong subtraction. Where regular subtraction removes from the first + machine any strings that are matched by the second machine, strong + subtraction removes any strings from the first that contain any strings of + the second as a substring. Strong subtraction is syntactic sugar for + expr1 - ( any* expr2 any* ). + -Eliminated the use of priorities from the examples. Replaced with + subtraction, guarded concatenation and longest-match kleene star. + -Did some initial work on supporting conditional transitions. Far from + complete and very buggy. This code will only be active when conditionals are + used. + +Ragel 5.3 - Jan 27, 2006 +======================== + -Added missing semi-colons that cause the build to fail when using older + versions of Bison. + -Fix for D code: if the contents of an fexec is a single word, the generated + code will get interpreted as a C-style cast. Adding two brackets prevents + this. Can now turn eliminate the "access this.;" in cppscan5 that was used to + get around this problem. + -Improved some of the tag names in the intermediate format. + -Added unsigned long to the list of supported alphabet types. + -Added ids of actions and action lists to XML intermediate format. Makes it + more human readable. + -Updated to latest Aapl package. + +Ragel 5.2 - Jan 6, 2006 +======================== + -Ragel emits an error if the target of fentry, fcall, fgoto or fnext is inside + a longest match operator, or if an action embedding in a longest match + machine uses fcall. The fcall command can still be used in pattern actions. + -Made improvements to the clang, rlscan, awkemu and cppscan examples. + -Some fixes to generated label names: they should all be prefixed with _. + -A fix to the Vim syntax highlighting script was made + -Many fixes and updates to the documentation. All important features and + concepts are now documented. A second chapter describing Ragel's use + was added. + +Ragel 5.1 - Dec 22, 2005 +======================== + -Fixes to the matching of section delimiters in Vim syntax file. + -If there is a longest match machine, the tokend var is now initialized by + write init. This is not necessary for correct functionality, however + prevents compiler warnings. + -The rlscan example was ported to the longest match operator and changed to + emit XML data. + -Fix to the error handling in the frontend: if there are errors in the lookup + of names at machine generation time then do not emit anything. + -If not compiling the full machine in the frontend (by using -M), avoid + errors and segfaults caused by names that are not part of the compiled + machine. + -Longest match bug fix: need to init tokstart when returing from fsm calls + that are inside longest match actions. + -In Graphviz drawing, the arrow into the start state is not a real + transition, do not draw to-state actions on the label. + -A bug fix to the handling of non-tag data within an XML tag was made. + -Backend exit value fixed: since the parser now accepts nothing so as to + avoid a redundant parse error when the frontend dies, we must force an + error. The backend should now be properly reporting errors. + -The longest match machine now has it's start state set final. An LM machine + is in a final state when it has not matched anything, when it has matched + and accepted a token and is ready for another, and when it has matched a + token but is waiting for some lookahead before determining what to do about + it (similar to kleene star). + -Element statement removed from some tests. + -Entry point names are propagated to the backend and used to label the entry + point arrows in Graphviz output. + +Ragel 5.0 - Dec 17, 2005 +======================== + (additional details in V5 release notes) + -Ragel has been split into two executables: A frontend which compiles + machines and emits them in an XML format, and a backend which generates code + or a Graphviz dot file from the XML input. The purpose of this split is to + allow Ragel to interface with other tools by means of the XML intermediate + format and to reduce complexity by strictly separating the previously + entangled phases. The intermediate format will provide a better platform + inspecting compiled machines and for extending Ragel to support other host + languages. + -The host language interface has been reduced significantly. Ragel no longer + expects the machine to be implemented as a structure or class and does not + generate functions corresponding to initialization, execution and EOF. + Instead, Ragel just generates the code of these components, allowing all of + them to be placed in a single function if desired. The user specifies a + machine in the usual manner, then indicates at which place in the program + text the state machine code is to be generated. This is done using the write + statement. It is possible to specify to Ragel how it should access the + variables it needs (such as the current state) using the access statement. + -The host language embedding delimiters have been changed. Single line + machines start with '%%' and end at newline. Multiline machines start with + '%%{' and end with '}%%'. The machine name is given with the machine + statement at the very beginning of the specification. This purpose of this + change is to make it easier separate Ragel code from the host language. This + will ease the addition of supported host languages. + -The structure and class parsing which was previously able to extract a + machine's name has been removed since this feature is dependent on the host + language and inhibits the move towards a more language-independent frontend. + -The init, element and interface statements have been made obsolete by the + new host language interface and have been removed. + -The fexec action statement has been changed to take only the new position to + move to. This statement is more useful for moving backwards and reparsing + input than for specifying a whole new buffer entirely and has been shifted + to this new use. Giving it only one argument also simplifies the parsing of + host code embedded in a Ragel specification. This will ease the addition of + supported host languages. + -Introduced the fbreak statement, which allows one to stop processing data + immediately. The machine ends up in the state that the current transition + was to go to. The current character is not changed. + -Introduced the noend option for writing the execute code. This inhibits + checking if we have reached pe. The machine will run until it goes into the + error state or fbreak is hit. This allows one to parse null-terminate + strings without first computing the length. + -The execute code now breaks out of the processing loop when it moves into + the error state. Previously it would run until pe was hit. Breaking out + makes the noend option useful when an error is encountered and allows + user code to determine where in the input the error occured. It also + eliminates needlessly iterating the input buffer. + -Introduced the noerror, nofinal and noprefix options for writing the machine + data. The first two inhibit the writing of the error state and the + first-final state should they not be needed. The noprefix eliminates the + prefixing of the data items with the machine name. + -Support for the D language has been added. This is specified in the backend + with the -D switch. + -Since the new host language interface has been reduced considerably, Ragel + no longer needs to distinguish between C-based languages. Support for C, C++ + and Objective-C has been folded into one option in the backend: -C + -The code generator has been made independent of the languages that it + supports by pushing the language dependent apsects down into the lower + levels of the code generator. + -Many improvements to the longest match construction were made. It is no + longer considered experimental. A longest match machine must appear at the + top level of a machine instantiation. Since it does not generate a pure + state machine (it may need to backtrack), it cannot be used as an operand to + other operators. + -References to the current character and current state are now completely + banned in EOF actions. + +Ragel 4.2 - Sep 16, 2005 +======================== + (additional details in V4 release notes) + -Fixed a bug in the longest match operator. In some states it's possible that + we either match a token or match nothing at all. In these states we need to + consult the LmSwitch on error so it must be prepared to execute an error + handler. We therefore need to init act to this error value (which is zero). + We can compute if we need to do this and the code generator emits the + initialization only if necessary. + -Changed the definition of the token end of longest match actions. It now + points to one past the last token. This makes computing the token length + easier because you don't have to add one. The longest match variables token + start, action identifier and token end are now properly initialized in + generated code. They don't need to be initialized in the user's code. + -Implemented to-state and from-state actions. These actions are executed on + transitions into the state (after the in transition's actions) and on + transitions out of the state (before the out transition's actions). See V4 + release notes for more information. + -Since there are no longer any action embedding operators that embed both on + transitions and on EOF, any actions that exist in both places will be there + because the user has explicitly done so. Presuming this case is rare, and + with code duplication in the hands of the user, we therefore give the EOF + actions their own action switch in the finish() function. This is further + motivated by the fact that the best solution is to do the same for to-state + and from-state actions in the main loop. + -Longest match actions can now be specified using a named action. Since a + word following a longest match item conflicts with the concatenation of a + named machine, the => symbol must come immediately before a named action. + -The longest match operator permits action and machine definitions in the + middle of a longest match construction. These are parsed as if they came + before the machine definition they are contained in. Permitting action and + machine definitions in a longest match construction allows objects to be + defined closer to their use. + -The longest match operator can now handle longest match items with no + action, where previously Ragel segfaulted. + -Updated to Aapl post 2.12. + -Fixed a bug in epsilon transition name lookups. After doing a name lookup + the result was stored in the parse tree. This is wrong because if a machine + is used more than once, each time it may resolve to different targets, + however it will be stored in the same place. We now store name resolutions + in a separated data structure so that each walk of a parse tree uses the + name resolved during the corresponding walk in the name lookup pass. + -The operators used to embed context and actions into states have been + modified. The V4 release notes contain the full details. + -Added zlen builtin machine to represent the zero length machine. Eventually + the name "null" will be phased out in favour of zlen because it is unclear + whether null matches the zero length string or if it does not match any + string at all (as does the empty builtin). + -Added verbose versions of action, context and priority embedding. See the V4 + release notes for the full details. A small example: + machine <- all exec { foo(); } <- final eof act1 + -Bugfix for machines with epsilon ops, but no join operations. I had + wrongfully assumed that because epsilon ops can only increase connectivity, + that no states are ever merged and therefore a call to fillInStates() is not + necessary. In reality, epsilon transitions within one machine can induce the + merging of states. In the following, state 2 follows two paths on 'i': + main := 'h' -> i 'i h' i: 'i'; + -Changed the license of the guide from a custom "do not propagate modified + versions of this document" license to the GPL. + +Ragel 4.1 - Jun 26, 2005 +======================== + (additional details in V4 release notes) + -A bug in include processing was fixed. Surrounding code in an include file + was being passed through to the output when it should be ignored. Includes + are only for including portions of another machine into he current. This + went unnoticed because all tested includes were wrapped in #ifndef ... + #endif directives and so did not affect the compilation of the file making + the include. + -Fixes were made to Vim syntax highlighting file. + -Duplicate actions are now removed from action lists. + -The character-level negation operator ^ was added. This operator produces a + machine that matches single characters that are not matched by the machine + it is applied to. This unary prefix operator has the same precedence level + as !. + -The use of + to specify the a positive literal number was discontinued. + -The parser now assigns the subtraction operator a higher precedence than + the negation of literal number. + +Ragel 4.0 - May 26, 2005 +======================== + (additional details in V4 release notes) + -Operators now strictly embed into a machine either on a specific class of + characters or on EOF, but never both. This gives a cleaner association + between the operators and the physical state machine entitites they operate + on. This change is made up of several parts: + 1. '%' operator embeds only into leaving characters. + 2. All global and local error operators only embed on error character + transitions, their action will not be triggerend on EOF in non-final + states. + 3. EOF action embedding operators have been added for all classes of states + to make up for functionality removed from other operators. These are + >/ $/ @/ %/. + 4. Start transition operator '>' no longer implicitly embeds into leaving + transtions when start state is final. + -Ragel now emits warnings about the improper use of statements and values in + action code that is embedded as an EOF action. Warnings are emitted for fpc, + fc, fexec, fbuf and fblen. + -Added a longest match construction operator |* machine opt-action; ... *|. + This is for repetition where an ability to revert to a shorter, previously + matched item is required. This is the same behaviour as flex and re2c. The + longest match operator is not a pure FSM construction, it introduces + transitions that implicitly hold the current character or reset execution to + a previous location in the input. Use of this operator requires the caller + of the machine to occasionally hold onto data after a call to the exectute + routine. Use of machines generated with this operator as the input to other + operators may have undefined results. See examples/cppscan for an example. + This is very experimental code. + -Action ids are only assigned to actions that are referenced in the final + constructed machine, preventing gaps in the action id sequence. Previously + an action id was assigned if the action was referenced during parsing. + -Machine specifications now begin with %% and are followed with an optional + name and either a single Ragel statement or a sequence of statements + enclosed in {}. + -Ragel no longer generates the FSM's structure or class. It is up to the user + to declare the structure and to give it a variable named curs of type + integer. If the machine uses the call stack the user must also declare a + array of integers named stack and an integer variable named top. + -In the case of Objective-C, Ragel no longer generates the interface or + implementation directives, allowing the user to declare additional methods. + -If a machine specification does not have a name then Ragel tries to find a + name for it by first checking if the specification is inside a struct, class + or interface. If it is not then it uses the name of the previous machine + specification. If still no name is found then this is an error. + -Fsm specifications now persist in memory and statements accumulate. + -Ragel now has an include statement for including the statements of a machine + spec in another file (perhaps because it is the corresponding header file). + The include statement can also be used to draw in the statements of another + fsm spec in the current file. + -The fstack statement is now obsolete and has been removed. + -A new statement, simply 'interface;', indicates that ragel should generate + the machine's interface. If Ragel sees the main machine it generates the + code sections of the machine. Previously, the header portion was generated + if the (now removed) struct statement was found and code was generated if + any machine definition was found. + -Fixed a bug in the resolution of fsm name references in actions. The name + resolution code did not recurse into inline code items with children + (fgoto*, fcall*, fnext*, and fexec), causing a segfault at code generation + time. + -Cleaned up the code generators. FsmCodeGen was made into a virtual base + class allowing for the language/output-style specific classes to inherit + both a language specific and style-specific base class while retaining only + one copy of FsmCodeGen. Language specific output can now be moved into the + language specific code generators, requiring less duplication of code in the + language/output-style specific leaf classes. + -Fixed bugs in fcall* implementation of IpgGoto code generation. + -If the element type has not been defined Ragel now uses a constant version + of the alphtype, not the exact alphtype. In most cases the data pointer of + the execute routine should be const. A non-const element type can still be + defined with the element statement. + -The fc special value now uses getkey for retrieving the current char rather + than *_p, which is wrong if the element type is a structure. + -User guide converted to TeX and updated for new 4.0 syntax and semantics. + +Ragel 3.7 - Oct 31, 2004 +======================== + -Bug fix: unreferenced machine instantiations causing segfault due to name + tree and parse tree walk becomming out of syncronization. + -Rewrote representation of inline code blocks using a tree data structure. + This allows special keywords such as fbuf to be used as the operatands of + other fsm commands. + -Documentation updates. + -When deciding whether or not to generate machine instantiations, search the + entire name tree beneath the instantiation for references, not just the + root. + -Removed stray ';' in keller2.rl + -Added fexec for restarting the machine with new buffer data (state stays the + same), fbuf for retrieving the the start of the buf, and fblen for + retrieving the orig buffer length. + -Implemented test/cppscan2 using fexec. This allows token emitting and restart + to stay inside the execute routine, instead of leaving and re-entering on + every token. + -Changed examples/cppscan to use fexec and thereby go much faster. + -Implemented flex and re2c versions of examples/cppscan. Ragel version + goes faster than flex version but not as fast as re2c version. + -Merged in Objective-C patch from Erich Ocean. + -Turned off syncing with stdio in C++ tests to make them go faster. + -Renamed C++ code generaion classes with the Cpp Prefix instead of CC to make + them easier to read. + -In the finish function emit fbuf as 0 cast to a pointer to the element type + so it's type is not interpreted as an integer. + -The number -128 underflows char alphabets on some architectures. Removed + uses of it in tests. + -Disabled the keller2 test because it causes problems on many architectures + due to its large size and compilation requirements. + +Ragel 3.6 - Jul 10, 2004 +======================== + -Many documentation updates. + -When resolving names, return a set of values so that a reference in an + action block that is embedded more than once won't report distinct entry + points that are actually the same. + -Implemented flat tables. Stores a linear array of indicies into the + transition array and only a low and high key value. Faster than binary + searching for keys but not usable for large alphabets. + -Fixed bug in deleting of transitions leftover from converstion from bst to + list implementation of transitions. Other code cleanup. + -In table based output calculate the cost of using an index. Don't use if + cheaper. + -Changed fstate() value available in init and action code to to fentry() to + reflect the fact that the values returned are intended to be used as targets + in fgoto, fnext and fcall statements. The returned state is not a unique + state representing the label. There can be any number of states representing + a label. + -Added keller2 test, C++ scanning tests and C++ scanning example. + -In table based output split up transitions into targets and actions. This + allows actions to be omitted. + -Broke the components of the state array into separate arrays. Requires + adding some fields where they could previously be omitted, however allows + finer grained control over the sizes of items and an overal size reduction. + Also means that state numbers are not an offset into the state array but + instead a sequence of numbers, meaning the context array does not have any + wasted bits. + -Action lists and transition also have their types chosen to be the smallest + possible for accomodating the contained values. + -Changed curs state stored in fsm struct from _cs to curs. Keep fsm->curs == + -1 while in machine. Added tests curs1 and curs2. + -Implemented the notion of context. Context can be embedded in states using + >:, $:, @: and %: operators. These embed a named context into start states, + all states, non-start/non-final and final states. If the context is declared + using a context statment + context name; + then the context can be quered for any state using fsm_name_ctx_name(state) + in C code and fsm_name::ctx_name(state) in C++ code. This feature makes it + possible to determine what "part" of the machine is currently active. + -Fixed crash on machine generation of graphs with no final state. If there + is no reference to a final state in a join operation, don't generate one. + -Updated Vim sytax: added labels to inline code, added various C++ keywords. + Don't highlight name separations as labels. Added switch labels, improved + alphtype, element and getkey. + -Fixed line info in error reporting of bad epsilon trans. + -Fixed fstate() for tab code gen. + -Removed references to malloc.h. + +Ragel 3.5 - May 29, 2004 +======================== + -When parse errors occur, the partially generated output file is deleted and + an non-zero exit status is returned. + -Updated Vim syntax file. + -Implemented the setting of the element type that is passed to the execute + routine as well as method for specifying how ragel should retrive the key + from the element type. This lets ragel process arbitrary structures inside + of which is the key that is parsed. + element struct Element; + getkey fpc->character; + -The current state is now implemented with an int across all machines. This + simplifies working with current state variables. For example this allows a + call stack to be implemented in user code. + -Implemented a method for retrieving the current state, the target state, and + any named states. + fcurs -retrieve the current state + ftargs -retrieve the target state + fstate(name) -retrieve a named state. + -Implemented a mechanism for jumping to and calling to a state stored in a + variable. + fgoto *<expr>; -goto the state returned by the C/C++ expression. + fcall *<expr>; -call the state returned by the C/C++ expression. + -Implemented a mechanism for specifying the next state without immediately + transfering control there (any code following statement is executed). + fnext label; -set the state pointed to by label as the next state. + fnext *<expr>; -set the state returned by the C/C++ expression as the + next. + -Action references are determined from the final machine instead of during + the parse tree walk. Some actions can be referenced in the parse tree but not + show up in the final machine. Machine analysis is now done based on this new + computation. + -Named state lookup now employs a breadth-first search in the lookup and + allows the user to fully qualify names, making it possible to specify + jumps/calls into parts of the machine deep in the name hierarchy. Each part + of name (separated by ::) employs a breadth first search from it's starting + point. + -Name references now must always refer to a single state. Since references to + multiple states is not normally intended, it no longer happens + automatically. This frees the programmer from thinking about whether or not + a state reference is unique. It also avoids the added complexity of + determining when to merge the targets of multiple references. The effect of + references to multiple states can be explicitly created using the join + operator and epsilon transitions. + -M option was split into -S and -M. -S specifies the machine spec to generate + for graphviz output and dumping. -M specifies the machine definition or + instantiation. + -Machine function parameters are now prefixed with and underscore to + avoid the hiding of class members. + +Ragel 3.4 - May 8, 2004 +======================= + -Added the longest match kleene star operator **, which is synonymous + with ( ( <machine> ) $0 %1 ) *. + -Epsilon operators distinguish between leaving transitions (going to an + another expression in a comma separated list) and non-leaving transitions. + Leaving actions and priorities are appropriately transferred. + -Relative priority of following ops changed to: + 1. Action/Priority + 2. Epsilon + 3. Label + If label is done first then the isolation of the start state in > operators + will cause the label to point to the old start state that doesn't have the + new action/priority. + -Merged >! and >~, @! and @~, %! and %~, and $! and $~ operators to have one + set of global error action operators (>!, @!, %! and $!) that are invoked on + error by unexpected characters as well as by unexepected EOF. + -Added the fpc keyword for use in action code. This is a pointer to the + current character. *fpc == fc. If an action is invoked on EOF then fpc == 0. + -Added >^, @^, %^, and $^ local error operators. Global error operators (>!, + @!, $!, and %!) cause actions to be invoked if the final machine fails. + Local error actions cause actions to be invoked if if the current machine + fails. + -Changed error operators to mean embed global/local error actions in: + >! and !^ -the start state. + @! and @^ -states that are not the start state and are not final. + %! and %^ -final states. + $! and $^ -all states. + -Added >@! which is synonymous >! then @! + -Added >@^ which is synonymous >^ then @^ + -Added @%! which is synonymous @! then %! + -Added @%^ which is synonymous >^ then @^ + -FsmGraph representation of transition lists was changed from a mapping of + alphabet key -> transition objects using a BST to simply a list of + transition objects. Since the transitions are no longer divided by + single/range, the fast finding of transition objects by key is no longer + required functionality and can be eliminated. This new implementation uses + the same amount of memory however causes less allocations. It also make more + sense for supporting error transitions with actions. Previously an error + transition was represented by a null value in the BST. + -Regular expression ranges are checked to ensure that lower <= upper. + -Added printf-like example. + -Added atoi2, erract2, and gotcallret to the test suite. + -Improved build test to support make -jN and simplified the compiling and + running of tests. + +Ragel 3.3 - Mar 7, 2004 +======================= + -Portability bug fixes were made. Minimum and maximum integer values are + now taken from the system. An alignment problem on 64bit systems + was fixed. + +Ragel 3.2 - Feb 28, 2004 +======================== + -Added a Vim syntax file. + -Eliminated length var from generated execute code in favour of an end + pointer. Using length requires two variables be read and written. Using an + end pointer requires one variable read and written and one read. Results in + more optimizable code. + -Minimization is now on by default. + -States are ordered in output by depth first search. + -Bug in minimization fixed. States were not being distinguished based on + error actions. + -Added null and empty builtin machines. + -Added EOF error action operators. These are >~, >@, $~, and %~. EOF error + operators embed actions to take if the EOF is seen and interpreted as an + error. The operators correspond to the following states: + -the start state + -any state with a transition to a final state + -any state with a transiion out + -a final state + -Fixed bug in generation of unreference machine vars using -M. Unreferenced + vars don't have a name tree built underneath when starting from + instantiations. Need to instead build the name tree starting at the var. + -Calls, returns, holds and references to fc in out action code are now + handled for ipgoto output. + -Only actions referenced by an instantiated machine expression are put into + the action index and written out. + -Added rlscan, an example that lexes Ragel input. + +Ragel 3.1 - Feb 18, 2004 +======================== + -Duplicates in OR literals are removed and no longer cause an assertion + failure. + -Duplicate entry points used in goto and call statements are made into + deterministic entry points. + -Base FsmGraph code moved from aapl into ragel, as an increasing amount + of specialization is required. Too much time was spent attempting to + keep it as a general purpose template. + -FsmGraph code de-templatized and heirarchy squashed to a single class. + -Single transitions taken out of FsmGraph code. In the machine construction + stage, transitions are now implemented only with ranges and default + transtions. This reduces memory consumption, simplifies code and prevents + covered transitions. However it requires the automated selection of single + transitions to keep goto-driven code lean. + -Machine reduction completely rewritten to be in-place. As duplicate + transitions and actions are found and the machine is converted to a format + suitable for writing as C code or as GraphViz input, the memory allocated + for states and transitions is reused, instead of newly allocated. + -New reduction code consolodates ranges, selects a default transition, and + selects single transitions with the goal of joining ranges that are split by + any number of single characters. + -Line directive changed from "# <num> <file>" to the more common format + "#line <num> <file>". + -Operator :! changed to @!. This should have happened in last release. + -Added params example. + +Ragel 3.0 - Jan 22, 2004 +======================== + -Ragel now parses the contents of struct statements and action code. + -The keyword fc replaces the use of *p to reference the current character in + action code. + -Machine instantiations other than main are allowed. + -Call, jump and return statements are now available in action code. This + facility makes it possible to jump to an error handling machine, call a + sub-machine for parsing a field or to follow paths through a machine as + determined by arbitrary C code. + -Added labels to the language. Labels can be used anywhere in a machine + expression to define an entry point. Also references to machine definitions + cause the implicit creation of a label. + -Added epsilon transitions to the language. Epsilon operators may reference + labels in the current name scope resolved when join operators are evaluated + and at the root of the expression tree of machine assignment/instantiation. + -Added the comma operator, which joins machines together without drawing any + transitions between them. This operator is useful in combination with + labels, the epsilon operator and user code transitions for defining machines + using the named state and transition list paradigm. It is also useful for + invoking transitions based on some analysis of the input or on the + environment. + -Added >!, :!, $!, %! operators for specifying actions to take should the + machine fail. These operators embed actions to execute if the machine + fails in + -the start state + -any state with a transition to a final state + -any state with a transiion out + -a final state + The general rule is that if an action embedding operator embeds an action + into a set of transitions T, then the error-counterpart with a ! embeds an + action into the error transition taken when any transition T is a candidate, + but does not match the input. + -The finishing augmentation operator ':' has been changed to '@'. This + frees the ':' symbol for machine labels and avoids hacks to the parser to + allow the use of ':' for both labels and finishing augmentations. The best + hack required that label names be distinct from machine definition names as + in main := word : word; This restriction is not good because labels are + local to the machine that they are used in whereas machine names are global + entities. Label name choices should not be restricted by the set of names + that are in use for machines. + -Named priority syntax now requires parenthesis surrounding the name and + value pair. This avoids grammar ambiguities now that the ',' operator has + been introduced and makes it more clear that the name and value are an + asscociated pair. + -Backslashes are escaped in line directive paths. + +Ragel 2.2 - Oct 6, 2003 +======================= + -Added {n}, {,n}, {n,} {n,m} repetition operators. + <expr> {n} -- exactly n repetitions + <expr> {,n} -- zero to n repetitions + <expr> {n,} -- n or more repetitions + <expr> {n,m} -- n to m repetitions + -Bug in binary search table in Aapl fixed. Fixes crashing on machines that + add to action tables that are implicitly shared among transitions. + -Tests using obsolete minimization algorithms are no longer built and run by + default. + -Added atoi and concurrent from examples to the test suite. + +Ragel 2.1 - Sep 22, 2003 +======================== + -Bug in priority comparison code fixed. Segfaulted on some input with many + embedded priorities. + -Added two new examples. + +Ragel 2.0 - Sep 7, 2003 +======================= + -Optional (?), One or More (+) and Kleene Star (*) operators changed from + prefix to postfix. Rationale is that postfix version is far more common in + regular expression implementations and will be more readily understood. + -All priority values attached to transitions are now accompanied by a name. + Transitions no longer have default priority values of zero assigned + to them. Only transitions that have different priority values assigned + to the same name influence the NFA-DFA conversion. This scheme reduces + side-effects of priorities. + -Removed the %! statement for unsetting pending out priorities. With + named priorities, it is not necessary to clear the priorities of a + machine with $0 %! because non-colliding names can be used to avoid + side-effects. + -Removed the clear keyword, which was for removing actions from a machine. + Not required functionality and it is non-intuitive to have a language + feature that undoes previous definitions. + -Removed the ^ modifier to repetition and concatenation operators. This + undocumented feature prevented out transitions and out priorities from being + transfered from final states to transitions leaving machines. Not required + functionality and complicates the language unnecessarily. + -Keyword 'func' changed to 'action' as a part of the phasing out of the term + 'function' in favour of 'action'. Rationale is that the term 'function' + implies that the code is called like a C function, which is not necessarily + the case. The term 'action' is far more common in state machine compiler + implementations. + -Added the instantiation statement, which looks like a standard variable + assignment except := is used instead of =. Instantiations go into the + same graph dictionary as definitions. In the the future, instantiations + will be used as the target for gotos and calls in action code. + -The main graph should now be explicitly instantiated. If it is not, + a warning is issued. + -Or literal basic machines ([] outside of regular expressions) now support + negation and ranges. + -C and C++ interfaces lowercased. In the C interface an underscore now + separates the fsm machine and the function name. Rationale is that lowercased + library and generated routines are more common. + C output: + int fsm_init( struct clang *fsm ); + int fsm_execute( struct clang *fsm, char *data, int dlen ); + int fsm_finish( struct clang *fsm ); + C++ output: + int fsm::init( ); + int fsm::execute( char *data, int dlen ); + int fsm::finish( ); + -Init, execute and finish all return -1 if the machine is in the error state + and can never accept, 0 if the machine is in a non-accepting state that has a + path to a final state and 1 if the machine is in an accepting state. + -Accept routine eliminated. Determining whether or not the machine accepts is + done by examining the return value of the finish routine. + -In C output, fsm structure is no longer a typedef, so referencing requires + the struct keyword. This is to stay in line with C language conventions. + -In C++ output, constructor is no longer written by ragel. As a consequence, + init routine is not called automatically. Allows constructor to be supplied + by user as well as the return value of init to be examined without calling it + twice. + -Static start state and private structures are taken out of C++ classes. + +Ragel 1.5.4 - Jul 14, 2003 +========================== + -Workaround for building with bison 1.875, which produces an + optimization that doesn't build with newer version gcc. + +Ragel 1.5.3 - Jul 10, 2003 +========================== + -Fixed building with versions of flex that recognize YY_NO_UNPUT. + -Fixed version numbers in ragel.spec file. + +Ragel 1.5.2 - Jul 7, 2003 +========================= + -Transition actions and out actions displayed in the graphviz output. + -Transitions on negative numbers handled in graphviz output. + -Warning generated when using bison 1.875 now squashed. + +Ragel 1.5.1 - Jun 21, 2003 +========================== + -Bugs fixed: Don't delete the output objects when writing to standard out. + Copy mem into parser buffer with memcpy, not strcpy. Fixes buffer mem errror. + -Fixes for compiling with Sun WorkShop 6 compilers. + +Ragel 1.5.0 - Jun 10, 2003 +========================== + -Line directives written to the output so that errors in the action code + are properly reported in the ragel input file. + -Simple graphviz dot file output format is supported. Shows states and + transitions. Does not yet show actions. + -Options -p and -f dropped in favour of -d output format. + -Added option -M for specifying the machine to dump with -d or the graph to + generate with -V. + -Error recovery implemented. + -Proper line and column number tracking implemented in the scanner. + -All action/function code is now embedded in the main Execute routine. Avoids + duplication of action code in the Finish routine and the need to call + ExecFuncs which resulted in huge code bloat. Will also allow actions to + modify cs when fsm goto, call and return is supported in action code. + -Fsm spec can have no statements, nothing will be generated. + -Bug fix: Don't accept ] as the opening of a .-. range a reg exp. + -Regular expression or set ranges (ie /[0-9]/) are now handled by the parser + and consequently must be well-formed. The following now generates a parser + error: /[+-]/ and must be rewritten as /[+\-]/. Also fixes a bug whereby ] + might be accepted as the opening of a .-. range causing /[0-9]-[0-9]/ to + parse incorrectly. + -\v, \f, and \r are now treated as whitespace in an fsm spec. + +Ragel 1.4.1 - Nov 19, 2002 +========================== + -Compile fixes. The last release (integer alphabets) was so exciting + that usual portability checks got bypassed. + +Ragel 1.4.0 - Nov 19, 2002 +========================== + -Arbitrary integer alphabets are now fully supported! A new language + construct: + 'alphtype <type>' added for specifying the type of the alphabet. Default + is 'char'. Possible alphabet types are: + char, unsigned char, short, unsigned short, int, unsigned int + -Literal machines specified in decimal format can now be negative when the + alphabet is a signed type. + -Literal machines (strings, decimal and hex) have their values checked for + overflow/underflow against the size of the alphabet type. + -Table driven and goto driven output redesigned to support ranges. Table + driven uses a binary search for locating single characters and ranges. Goto + driven uses a switch statement for single characters and nested if blocks for + ranges. + -Switch driven output removed due to a lack of consistent advantages. Most of + the time the switch driven FSM is of no use because the goto FSM makes + smaller and faster code. Under certain circumstances it can produce smaller + code than a goto driven fsm and be almost as fast, but some sporadic case + does not warrant maintaining it. + -Many warnings changed to errors. + -Added option -p for printing the final fsm before minimization. This lets + priorities be seen. Priorties are all reset to 0 before minimization. The + exiting option -f prints the final fsm after minimization. + -Fixed a bug in the clang test and example that resulted in redundant actions + being executed. + +Ragel 1.3.4 - Nov 6, 2002 +========================= + -Fixes to Chapter 1 of the guide. + -Brought back the examples and made them current. + -MSVC is no longer supported for compiling windows binaries because its + support for the C++ standard is frustratingly inadequate, it will cost money + to upgrade if it ever gets better, and MinGW is a much better alternative. + -The build system now supports the --host= option for building ragel + for another system (used for cross compiling a windows binary with MinGW). + -Various design changes and fixes towards the goal of arbitrary integer + alphabets and the handling of larger state machines were made. + -The new shared vector class is now used for action lists in transitions and + states to reduce memory allocations. + -An avl tree is now used for the reduction of transitions and functions of an + fsm graph before making the final machine. The tree allows better scalability + and performance by not requiring consecutively larger heap allocations. + -Final stages in the separation of fsm graph code from action embedding and + priority assignment is complete. Makes the base graph leaner and easier to reuse + in other projects (like Keller). + +Ragel 1.3.3 - Oct 22, 2002 +========================== + -More diagrams were added to section 1.7.1 of the user guide. + -FSM Graph code was reworked to spearate the regex/nfa/minimizaion graph + algorithms from the manipulation of state and transition properties. + -An rpm spec file from Cris Bailiff was added. This allows an rpm for ragel + to be built with the command 'rpm -ta ragel-x.x.x.tar.gz' + -Fixes to the build system and corresponding doc updates in the README. + -Removed autil and included the one needed source file directly in the top + level ragel directory. + -Fixed a bug that nullified the 20 times speedup in large compilations + claimed by the last version. + -Removed awk from the doc build (it was added with the last release -- though + not mentioned in the changelog). + -Install of man page was moved to the doc dir. The install also installs the + user guide to $(PREFIX)/share/doc/ragel/ + +Ragel 1.3.2 - Oct 16, 2002 +========================== + -Added option -v (or --version) to show version information. + -The subtract operator no longer removes transition data from the machine + being subtracted. This is left up to the user for the purpose of making it + possible to transfer transitions using subtract and also for speeding up the + subtract routine. Note that it is possible to explicitly clear transition + data before a doing a subtract. + -Rather severe typo bug fixed. Bug was related to transitions with higher + priorities taking precedence. A wrong ptr was being returned. It appears to + have worked most of the time becuase the old ptr was deleted and the new one + allocated immediatly after so the old ptr often pointed to the same space. + Just luck though. + -Bug in the removing of dead end paths was fixed. If the start state + has in transitions then those paths were not followed when finding states to + keep. Would result in non-dead end states being removed from the graph. + -In lists and in ranges are no longer maintained as a bst with the key as the + alphabet character and the value as a list of transitions coming in on that + char. There is one list for each of inList, inRange and inDefault. Now that + the required functionality of the graph is well known it is safe to remove + these lists to gain in speed and footprint. They shouldn't be needed. + -IsolateStartState() runs on modification of start data only if the start + state is not already isolated, which is now possible with the new in list + representation. + -Concat, Or and Star operators now use an approximation to + removeUnreachableStates that does not require a traversal of the entire + graph. This combined with an 'on-the-fly' management of final bits and final + state status results is a dramatic speed increase when compiling machines + that use those operators heavily. The strings2 test goes 20 times faster. + -Before the final minimization, after all fsm operations are complete, + priority data is reset which enables better minimization in cases where + priorities would otherwise separate similar states. + +Ragel 1.3.1 - Oct 2, 2002 +========================= + -Range transitions are now used to implement machines made with /[a-z]/ and + the .. operator as well as most of the builtin machines. The ranges are not + yet reflected in the output code, they are expanded as if they came from the + regular single transitions. This is one step closer to arbitrary integer + output. + -The builtin machine 'any' was added. It is equiv to the builtin extend, + matching any characters. + -The builtin machine 'cntrl' now includes newline. + -The builtin machine 'space' now includes newline. + -The builtin machine 'ascii' is now the range 0-127, not all characters. + -A man page was written. + -A proper user guide was started. Chapter 1: Specifying Ragel Programs + was written. It even has some diagrams :) + +Ragel 1.3.0 - Sep 4, 2002 +========================= + -NULL keyword no longer used in table output. + -Though not yet in use, underlying graph structure changed to support range + transitions. As a result, most of the code that walks transition lists is now + implemented with an iterator that hides the complexity of the transition + lists and ranges. Range transitions will be used to implement /[a-z]/ style + machines and machines made with the .. operator. Previously a single + transition would be used for each char in the range, which is very costly. + Ranges eliminate much of the space complexity and allow for the .. operator + to be used with very large (integer) alphabets. + -New minimization similar to Hopcroft's alg. It does not require n^2 space and + runs close to O(n*log(n)) (an exact analysis of the alg is very hard). It is + much better than the stable and approx minimization and obsoletes them both. + An exact implementation of Hopcroft's alg is desirable but not possible + because the ragel implementation does not assume a finite alphabet, which + Hopcroft's requires. Ragel will support arbitrary integer alphabets which + must be treated as an infinite set for implementation considerations. + -New option -m using above described minimization to replace all previous + minimization options. Old options sill work but are obsolete and not + advertised with -h. + -Bug fixed in goto style output. The error exit set the current state to 0, + which is actually a valid state. If the machine was entered again it would go + into the first state, very wrong. If the first state happened to be final then + an immediate finish would accept when in fact it should fail. + -Slightly better fsm minimization now capable due to clearing of the + transition ordering numbers just prior to minimization. + +Ragel 1.2.2 - May 25, 2002 +========================== + -Configuration option --prefix now works when installing. + -cc file extension changed to cpp for better portability. + -Unlink of output file upon error no longer happens, removes dependency on + unlink system command. + -All multiline strings removed: not standard c++. + -Awk build dependency removed. + -MSVC 6.0 added to the list of supported compilers (with some tweaking of + bison and flex output). + +Ragel 1.2.1 - May 13, 2002 +========================== + -Automatic dependencies were fixed, they were not working correctly. + -Updated AUTHORS file to reflect contributors. + -Code is more C++ standards compliant: compiles with g++ 3.0 + -Fixed bugs that only showed up in g++ 3.0 + -Latest (unreleased) Aapl. + -Configuration script bails out if bison++ is installed. Ragel will not + compile with bison++ because it is coded in c++ and bison++ automatically + generates a c++ parser. Ragel uses a c-style bison parser. + +Ragel 1.2.0 - May 3, 2002 +========================= + -Underlying graph structure now supports default transitions. The result is + that a transition does not need to be made for each char of the alphabet + when making 'extend' or '/./' machines. Ragel compiles machines that + use the aforementioned primitives WAY faster. + -The ugly hacks needed to pick default transitions now go away due to + the graph supporting default transitions directly. + -If -e is given, but minimization is not turned on, print a warning. + -Makefiles use automatic dependencies. + +Ragel 1.1.0 - Apr 15, 2002 +========================== + -Added goto fsm: much faster than any other fsm style. + -Default operator (if two machines are side by side with no operator + between them) is concatenation. First showed up in 1.0.4. + -The fsm machine no longer auotmatically builds the flat table for + transition indicies. Instead it keeps the key,ptr pair. In tabcodegen + the flat table is produced. This way very large alphabets with sparse + transitions will not consume large amounts of mem. This is also in prep + for fsm graph getting a default transition. + -Generated code contains a statement explicitly stating that ragel fsms + are NOT covered by the GPL. Technically, Ragel copies part of itself + to the output to make the generic fsm execution routine (for table driven + fsms only) and so the output could be considered under the GPL. But this + code is very trivial and could easlily be rewritten. The actual fsm data + is subject to the copyright of the source. To promote the use of Ragel, + a special exception is made for the part of the output copied from Ragel: + it may be used without restriction. + -Much more elegant code generation scheme is employed. Code generation + class members need only put the 'codegen' keyword after their 'void' type + in order to be automatically registerd to handle macros of the same name. + An awk script recognises this keyword and generates an appropriate driver. + -Ragel gets a test suite. + -Postfunc and prefunc go away because they are not supported by non + loop-driven fsms (goto, switch) and present duplicate functionality. + Universal funcs can be implemented by using $ operator. + -Automatic dependencies used in build system, no more make depend target. + -Code generation section in docs. + -Uses the latests aapl. + +Ragel 1.0.5 - Mar 3, 2002 +========================= + -Bugfix in SetErrorState that caused an assertion failure when compiling + simple machines that did not have full transition tables (and thus did + not show up on any example machines). Assertion failure did not occur + when using the switch statement code as ragel does not call SetErrorState + in that case. + -Fixed some missing includes, now compiles on redhat. + -Moved the FsmMachTrans Compare class out of FsmMachTrans. Some compilers + don't deal with nested classes in templates too well. + -Removed old unused BASEREF in fsmgraph and ragel now compiles using + egcs-2.91.66 and presumably SUNWspro. The baseref is no longer needed + because states do not support being elements in multiple lists. I would + rather be able to support more compilers than have this feature. + -Started a README with compilation notes. Started an AUTHORS file. + -Started the user documentation. Describes basic machines and operators. + +Ragel 1.0.4 - Mar 1, 2002 +========================= + -Ported to the version of Aapl just after 2.2.0 release. See + http://www.ragel.ca/aapl/ for details on aapl. + -Fixed a bug in the clang example: the newline machine was not stared. + -Added explanations to the clang and mailbox examples. This should + help people that want to learn the lanuage as the manual is far from + complete. + +Ragel 1.0.3 - Feb 2, 2002 +========================= + -Added aapl to the ragel tree. No longer requires you to download + and build aapl separately. Should avoid discouraging impatient users + from compiling ragel. + -Added the examples to the ragel tree. + -Added configure script checks for bison and flex. + -Fixed makefile so as not to die with newer versions of bison that + write the header of the parser to a .hh file. + -Started ChangeLog file. + +Ragel 1.0.2 - Jan 30, 2002 +========================== + -Bug fix in calculating highIndex for table based code. Was using + the length of out tranisition table rather than the value at the + end. + -If high/low index are at the limits, output a define in their place, + not the high/low values themselves so as not to cause compiler warnings. + -If the resulting machines don't have any indicies or functions, then + omit the empty unrefereced static arrays so as not to cause compiler + warnings about unused static vars. + -Fixed variable sized indicies support. The header cannot have any + reference to INDEX_TYPE as that info is not known at the time the header + data is written. Forces us to use a void * for pointers to indicies. In + the c++ versions we are forced to make much of the data non-member + static data in the code portion for the same reason. + +Ragel 1.0.1 - Jan 28, 2002 +========================== + -Exe name change from reglang to ragel. + -Added ftabcodegen output code style which uses a table for states and + transitions but uses a switch statement for the function execution. + -Reformatted options in usage dump to look better. + -Support escape sequences in [] sections of regular expressions. + +Ragel 1.0 - Jan 25, 2002 +======================== + -Initial release. diff --git a/contrib/tools/ragel6/DIST b/contrib/tools/ragel6/DIST new file mode 100644 index 0000000000..85d0524832 --- /dev/null +++ b/contrib/tools/ragel6/DIST @@ -0,0 +1,6 @@ +#!/bin/sh + +# Change to yes to enable building of parsers or manual. Reconfigure +# afterwards. +build_parsers=no; +build_manual=no; diff --git a/contrib/tools/ragel6/README b/contrib/tools/ragel6/README new file mode 100644 index 0000000000..d05b863e26 --- /dev/null +++ b/contrib/tools/ragel6/README @@ -0,0 +1,32 @@ + + Ragel State Machine Compiler -- README + ====================================== + +1. Build Requirements +--------------------- + + * Make + * g++ + +If you would like to modify Ragel and need to build Ragel's scanners and +parsers from the specifications then set "build_parsers=yes" the DIST file and +reconfigure. This variable is normally set to "no" in the distribution tarballs +and "yes" in version control. You will need the following programs: + + * ragel (the most recent version) + * kelbt (the most recent version) + +To build the user guide set "build_manual=yes" in the DIST file and +reconfigure. You will need the following extra programs: + + * fig2dev + * pdflatex + +2. Compilation and Installation +------------------------------- + +Ragel uses autoconf and automake. + +$ ./configure --prefix=PREFIX +$ make +$ make install diff --git a/contrib/tools/ragel6/TODO b/contrib/tools/ragel6/TODO new file mode 100644 index 0000000000..85c5440ff0 --- /dev/null +++ b/contrib/tools/ragel6/TODO @@ -0,0 +1,102 @@ +Guard against including a ragel file more than once (newsgroup). + +Line numbers in included files refer to the master file. This needs to be +fixed -- from Manoj. + +Remove old action embedding and condition setting syntax. + +fbreak should advance the current char. Depreciate fbreak and add + fctl_break; + fctl_return <expr>; + fctl_goto <label>; +This is needed for better support of pull scanners. + +Eliminate tokend, replace it with the marker variable and add ftokend/ftoklen. + +Add the accept action embedding operator: like eof but only for states that end +up final. Add the combined pending/accept action. This becomes a good idea when +eof action execution is moved into the main loop. + +Add a prefix operator which sets every state final. + +Minimization should remove a condition when the character allows both +the positive and negative sense of the condition. This happens in: +test_every_10_chars = ( ( c when test_len ) c{0,9} )**; +In this example there is non-determinsm that is killed by the priorities, but +since conditions are expanded before priorities are tested, many transitions +end up with test_len || !test_len. + +Should be possible to include scanner definitions in another scanner. + +Need an "entry name;" feature, causing name to get written out with the other +entry points in the data. + +Possibly bring back the old semantics of > in a new operator, or allow it to be +defined somehow. + +When priorities are embedded without a name, the name of the current machine is +used. Perhaps a unique name for each instance of the current machine should be +used instead. This idea could work well if applied to user-defined embeddings. + +User defined embeddings <-name(a1,a2,...). +User defined operators expr1 <name> expr2. + +Doesn't make make sense for [] to be the lambda (zero-length) machine. This +should be the empty set (the empty machine). But then would it be invalid in a +regular expression? + +The |> guarded operator and the <| guarded operator need to be added. + +An option to turn off the removal of duplicate actions might be useful for +analyzing unintentional nondeterminism. + +Might be a good idea to add in some protection against using up all of a +system's memory. This protection could then be removed by people when someone +is sure they want to use a lot of memory. + + +If a scanner can be optimized into a pure state machine, maybe permit it to be +referenced as a machine definition. Alternately: inline scanners with an +explicit exit pattern. + +The split codegen needs a profiler connected to a graph partitioning algorithm. + +Die a graceful death when rlcodegen -F receives large alphabets. + +It's not currently possible to have more than one machine in a single function +because of label conflicts. Labels should have a unique prefix. + +Emit a warning when a subtraction has no effect. + +Emit a warning when unnamed priorities are used in longest match machines. +These priorities may unexpectedly interact across longest-match items. Changing +the language such that unwated interaction cannot happen would require naming +longest-match items. + +Testing facilities: Quick easy way to query which strings are accepted. +Enumerate all accepted strings. From Nicholas Maxwell Lester. + +Add more examples, add more tests and write more documentation. + +A debugger would be nice. Ragel could emit a special debug version that +prompted for debug commands that allowed the user to step through the machine +and get details about where they are in their RL. + +A quick and easy alternative would be a trace code generation option. This +would open a trace file and list all the active machines at each step of the +input. + +Frontend should allow the redefinition of fsm section delimiters. + +Do more to obscure ragel's private variables. Just a leading underscore is not +enough. Maybe something more like __ri__. + +Some talk about capturing data: + +Separate tokstart/tokend from the backtracking. One var for preservation, +called preserve. Write delcarations; produces the necessary variables used by +ragel. Move pattern start pattern end concepts into the general? The +variables which may need to influence the preserve is dependent on the state. +States have a concept of which variables are in use. Can be used for length +restrictions. If there is an exit pattern, it is the explicit way out, +otherwise the start state and all final states are a way out. diff --git a/contrib/tools/ragel6/bin/ya.make b/contrib/tools/ragel6/bin/ya.make new file mode 100644 index 0000000000..3adf88c56b --- /dev/null +++ b/contrib/tools/ragel6/bin/ya.make @@ -0,0 +1,116 @@ +OWNER(pg g:cpp-contrib) + +PROGRAM(ragel6) + +VERSION(6.10) + +LICENSE(GPL-2.0) + +PEERDIR( + contrib/tools/ragel5/aapl +) + +ADDINCL( + contrib/tools/ragel6 +) + +NO_COMPILER_WARNINGS() + +NO_UTIL() + +CFLAGS( + -DHAVE_CONFIG_H +) + +SRCDIR(contrib/tools/ragel6) + +SRCS( + rlparse.cpp + rlscan.cpp +) + +JOIN_SRCS( + all_cd.cpp + cdcodegen.cpp + cdfflat.cpp + cdfgoto.cpp + cdflat.cpp + cdftable.cpp + cdgoto.cpp + cdipgoto.cpp + cdsplit.cpp + cdtable.cpp +) + +JOIN_SRCS( + all_cs.cpp + cscodegen.cpp + csfflat.cpp + csfgoto.cpp + csflat.cpp + csftable.cpp + csgoto.cpp + csipgoto.cpp + cssplit.cpp + cstable.cpp +) + +JOIN_SRCS( + all_fs.cpp + fsmap.cpp + fsmattach.cpp + fsmbase.cpp + fsmgraph.cpp + fsmmin.cpp + fsmstate.cpp +) + +JOIN_SRCS( + all_go.cpp + gocodegen.cpp + gofflat.cpp + gofgoto.cpp + goflat.cpp + goftable.cpp + gogoto.cpp + goipgoto.cpp + gotable.cpp + gotablish.cpp +) + +JOIN_SRCS( + all_ml.cpp + mlcodegen.cpp + mlfflat.cpp + mlfgoto.cpp + mlflat.cpp + mlftable.cpp + mlgoto.cpp + mltable.cpp +) + +JOIN_SRCS( + all_r.cpp + rbxgoto.cpp + redfsm.cpp + rubycodegen.cpp + rubyfflat.cpp + rubyflat.cpp + rubyftable.cpp + rubytable.cpp +) + +JOIN_SRCS( + all_other.cpp + common.cpp + dotcodegen.cpp + gendata.cpp + inputdata.cpp + javacodegen.cpp + main.cpp + parsedata.cpp + parsetree.cpp + xmlcodegen.cpp +) + +END() diff --git a/contrib/tools/ragel6/cdcodegen.cpp b/contrib/tools/ragel6/cdcodegen.cpp new file mode 100644 index 0000000000..9e784e3f58 --- /dev/null +++ b/contrib/tools/ragel6/cdcodegen.cpp @@ -0,0 +1,1018 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "cdcodegen.h" +#include "ragel.h" +#include "redfsm.h" +#include "gendata.h" +#include <sstream> +#include <string> +#include <assert.h> + + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + + +extern int numSplitPartitions; +extern bool noLineDirectives; + +void cdLineDirective( ostream &out, const char *fileName, int line ) +{ + if ( noLineDirectives ) + out << "/* "; + + /* Write the preprocessor line info for to the input file. */ + out << "#line " << line << " \""; + for ( const char *pc = fileName; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + out << "\\\\"; + else + out << *pc; + } + out << '"'; + + if ( noLineDirectives ) + out << " */"; + + out << '\n'; +} + +void FsmCodeGen::genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + cdLineDirective( out, filter->fileName, filter->line + 1 ); +} + + +/* Init code gen with in parameters. */ +FsmCodeGen::FsmCodeGen( ostream &out ) +: + CodeGenData(out) +{ +} + +unsigned int FsmCodeGen::arrayTypeSize( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + return arrayType->size; +} + +string FsmCodeGen::ARRAY_TYPE( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + + string ret = arrayType->data1; + if ( arrayType->data2 != 0 ) { + ret += " "; + ret += arrayType->data2; + } + return ret; +} + + +/* Write out the fsm name. */ +string FsmCodeGen::FSM_NAME() +{ + return fsmName; +} + +/* Emit the offset of the start state as a decimal integer. */ +string FsmCodeGen::START_STATE_ID() +{ + ostringstream ret; + ret << redFsm->startState->id; + return ret.str(); +}; + +/* Write out the array of actions. */ +std::ostream &FsmCodeGen::ACTIONS_ARRAY() +{ + out << "\t0, "; + int totalActions = 1; + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + /* Write out the length, which will never be the last character. */ + out << act->key.length() << ", "; + /* Put in a line break every 8 */ + if ( totalActions++ % 8 == 7 ) + out << "\n\t"; + + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) { + out << item->value->actionId; + if ( ! (act.last() && item.last()) ) + out << ", "; + + /* Put in a line break every 8 */ + if ( totalActions++ % 8 == 7 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +string FsmCodeGen::ACCESS() +{ + ostringstream ret; + if ( accessExpr != 0 ) + INLINE_LIST( ret, accessExpr, 0, false, false ); + return ret.str(); +} + + +string FsmCodeGen::P() +{ + ostringstream ret; + if ( pExpr == 0 ) + ret << "p"; + else { + ret << "("; + INLINE_LIST( ret, pExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::PE() +{ + ostringstream ret; + if ( peExpr == 0 ) + ret << "pe"; + else { + ret << "("; + INLINE_LIST( ret, peExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::vEOF() +{ + ostringstream ret; + if ( eofExpr == 0 ) + ret << "eof"; + else { + ret << "("; + INLINE_LIST( ret, eofExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::vCS() +{ + ostringstream ret; + if ( csExpr == 0 ) + ret << ACCESS() << "cs"; + else { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, csExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::TOP() +{ + ostringstream ret; + if ( topExpr == 0 ) + ret << ACCESS() + "top"; + else { + ret << "("; + INLINE_LIST( ret, topExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::STACK() +{ + ostringstream ret; + if ( stackExpr == 0 ) + ret << ACCESS() + "stack"; + else { + ret << "("; + INLINE_LIST( ret, stackExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::ACT() +{ + ostringstream ret; + if ( actExpr == 0 ) + ret << ACCESS() + "act"; + else { + ret << "("; + INLINE_LIST( ret, actExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::TOKSTART() +{ + ostringstream ret; + if ( tokstartExpr == 0 ) + ret << ACCESS() + "ts"; + else { + ret << "("; + INLINE_LIST( ret, tokstartExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::TOKEND() +{ + ostringstream ret; + if ( tokendExpr == 0 ) + ret << ACCESS() + "te"; + else { + ret << "("; + INLINE_LIST( ret, tokendExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string FsmCodeGen::GET_WIDE_KEY() +{ + if ( redFsm->anyConditions() ) + return "_widec"; + else + return GET_KEY(); +} + +string FsmCodeGen::GET_WIDE_KEY( RedStateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) + return "_widec"; + else + return GET_KEY(); +} + +string FsmCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + ret << "(*" << P() << ")"; + } + return ret.str(); +} + +/* Write out level number of tabs. Makes the nested binary search nice + * looking. */ +string FsmCodeGen::TABS( int level ) +{ + string result; + while ( level-- > 0 ) + result += "\t"; + return result; +} + +/* Write out a key from the fsm code gen. Depends on wether or not the key is + * signed. */ +string FsmCodeGen::KEY( Key key ) +{ + ostringstream ret; + if ( keyOps->isSigned || !hostLang->explicitUnsigned ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal() << 'u'; + return ret.str(); +} + +bool FsmCodeGen::isAlphTypeSigned() +{ + return keyOps->isSigned; +} + +bool FsmCodeGen::isWideAlphTypeSigned() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + return isAlphTypeSigned(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + return wideType->isSigned; + } +} + +string FsmCodeGen::WIDE_KEY( RedStateAp *state, Key key ) +{ + if ( state->stateCondList.length() > 0 ) { + ostringstream ret; + if ( isWideAlphTypeSigned() ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal() << 'u'; + return ret.str(); + } + else { + return KEY( key ); + } +} + +void FsmCodeGen::EOF_CHECK( ostream &ret ) +{ + ret << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + + testEofUsed = true; +} + + +void FsmCodeGen::EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ) +{ + /* The parser gives fexec two children. The double brackets are for D + * code. If the inline list is a single word it will get interpreted as a + * C-style cast by the D compiler. */ + ret << "{" << P() << " = (("; + INLINE_LIST( ret, item->children, targState, inFinish, false ); + ret << "))-1;}"; +} + +void FsmCodeGen::LM_SWITCH( ostream &ret, GenInlineItem *item, + int targState, int inFinish, bool csForced ) +{ + ret << + " switch( " << ACT() << " ) {\n"; + + bool haveDefault = false; + for ( GenInlineList::Iter lma = *item->children; lma.lte(); lma++ ) { + /* Write the case label, the action and the case break. */ + if ( lma->lmId < 0 ) { + ret << " default:\n"; + haveDefault = true; + } + else + ret << " case " << lma->lmId << ":\n"; + + /* Write the block and close it off. */ + ret << " {"; + INLINE_LIST( ret, lma->children, targState, inFinish, csForced ); + ret << "}\n"; + + ret << " break;\n"; + } + + if ( (hostLang->lang == HostLang::D || hostLang->lang == HostLang::D2) && !haveDefault ) + ret << " default: break;"; + + ret << + " }\n" + "\t"; +} + +void FsmCodeGen::SET_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = " << item->lmId << ";"; +} + +void FsmCodeGen::SET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + /* The tokend action sets tokend. */ + ret << TOKEND() << " = " << P(); + if ( item->offset != 0 ) + out << "+" << item->offset; + out << ";"; +} + +void FsmCodeGen::GET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + ret << TOKEND(); +} + +void FsmCodeGen::INIT_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << NULL_ITEM() << ";"; +} + +void FsmCodeGen::INIT_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = 0;"; +} + +void FsmCodeGen::SET_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << P() << ";"; +} + +void FsmCodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish, bool csForced ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << "{"; + INLINE_LIST( ret, item->children, targState, inFinish, csForced ); + ret << "}"; + } +} + + +/* Write out an inline tree structure. Walks the list and possibly calls out + * to virtual functions than handle language specific items in the tree. */ +void FsmCodeGen::INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish, bool csForced ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Text: + ret << item->data; + break; + case GenInlineItem::Goto: + GOTO( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Call: + CALL( ret, item->targState->id, targState, inFinish ); + break; + case GenInlineItem::Next: + NEXT( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Ret: + RET( ret, inFinish ); + break; + case GenInlineItem::PChar: + ret << P(); + break; + case GenInlineItem::Char: + ret << GET_KEY(); + break; + case GenInlineItem::Hold: + ret << P() << "--;"; + break; + case GenInlineItem::Exec: + EXEC( ret, item, targState, inFinish ); + break; + case GenInlineItem::Curs: + CURS( ret, inFinish ); + break; + case GenInlineItem::Targs: + TARGS( ret, inFinish, targState ); + break; + case GenInlineItem::Entry: + ret << item->targState->id; + break; + case GenInlineItem::GotoExpr: + GOTO_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::CallExpr: + CALL_EXPR( ret, item, targState, inFinish ); + break; + case GenInlineItem::NextExpr: + NEXT_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::LmSwitch: + LM_SWITCH( ret, item, targState, inFinish, csForced ); + break; + case GenInlineItem::LmSetActId: + SET_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokEnd: + SET_TOKEND( ret, item ); + break; + case GenInlineItem::LmGetTokEnd: + GET_TOKEND( ret, item ); + break; + case GenInlineItem::LmInitTokStart: + INIT_TOKSTART( ret, item ); + break; + case GenInlineItem::LmInitAct: + INIT_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokStart: + SET_TOKSTART( ret, item ); + break; + case GenInlineItem::SubAction: + SUB_ACTION( ret, item, targState, inFinish, csForced ); + break; + case GenInlineItem::Break: + BREAK( ret, targState, csForced ); + break; + } + } +} +/* Write out paths in line directives. Escapes any special characters. */ +string FsmCodeGen::LDIR_PATH( char *path ) +{ + ostringstream ret; + for ( char *pc = path; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + ret << "\\\\"; + else + ret << *pc; + } + return ret.str(); +} + +void FsmCodeGen::ACTION( ostream &ret, GenAction *action, int targState, + bool inFinish, bool csForced ) +{ + /* Write the preprocessor line info for going into the source file. */ + cdLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + ret << "\t{"; + INLINE_LIST( ret, action->inlineList, targState, inFinish, csForced ); + ret << "}\n"; +} + +void FsmCodeGen::CONDITION( ostream &ret, GenAction *condition ) +{ + ret << "\n"; + cdLineDirective( ret, condition->loc.fileName, condition->loc.line ); + INLINE_LIST( ret, condition->inlineList, 0, false, false ); +} + +string FsmCodeGen::ERROR_STATE() +{ + ostringstream ret; + if ( redFsm->errState != 0 ) + ret << redFsm->errState->id; + else + ret << "-1"; + return ret.str(); +} + +string FsmCodeGen::FIRST_FINAL_STATE() +{ + ostringstream ret; + if ( redFsm->firstFinState != 0 ) + ret << redFsm->firstFinState->id; + else + ret << redFsm->nextStateId; + return ret.str(); +} + +void FsmCodeGen::writeInit() +{ + out << " {\n"; + + if ( !noCS ) + out << "\t" << vCS() << " = " << START() << ";\n"; + + /* If there are any calls, then the stack top needs initialization. */ + if ( redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\t" << TOP() << " = 0;\n"; + + if ( hasLongestMatch ) { + out << + " " << TOKSTART() << " = " << NULL_ITEM() << ";\n" + " " << TOKEND() << " = " << NULL_ITEM() << ";\n" + " " << ACT() << " = 0;\n"; + } + out << " }\n"; +} + +string FsmCodeGen::DATA_PREFIX() +{ + if ( !noPrefix ) + return FSM_NAME() + "_"; + return ""; +} + +/* Emit the alphabet data type. */ +string FsmCodeGen::ALPH_TYPE() +{ + string ret = keyOps->alphType->data1; + if ( keyOps->alphType->data2 != 0 ) { + ret += " "; + ret += + keyOps->alphType->data2; + } + return ret; +} + +/* Emit the alphabet data type. */ +string FsmCodeGen::WIDE_ALPH_TYPE() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + ret = ALPH_TYPE(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + assert( wideType != 0 ); + + ret = wideType->data1; + if ( wideType->data2 != 0 ) { + ret += " "; + ret += wideType->data2; + } + } + return ret; +} + +void FsmCodeGen::STATE_IDS() +{ + if ( redFsm->startState != 0 ) + STATIC_VAR( "int", START() ) << " = " << START_STATE_ID() << "};\n"; + + if ( !noFinal ) + STATIC_VAR( "int" , FIRST_FINAL() ) << " = " << FIRST_FINAL_STATE() << "};\n"; + + if ( !noError ) + STATIC_VAR( "int", ERROR() ) << " = " << ERROR_STATE() << "};\n"; + + out << "\n"; + + if ( !noEntry && entryPointNames.length() > 0 ) { + for ( EntryNameVect::Iter en = entryPointNames; en.lte(); en++ ) { + STATIC_VAR( "int", DATA_PREFIX() + "en_" + *en ) << + " = " << entryPointIds[en.pos()] << "};\n"; + } + out << "\n"; + } +} + +void FsmCodeGen::writeStart() +{ + out << START_STATE_ID(); +} + +void FsmCodeGen::writeFirstFinal() +{ + out << FIRST_FINAL_STATE(); +} + +void FsmCodeGen::writeError() +{ + out << ERROR_STATE(); +} + +/* + * Language specific, but style independent code generators functions. + */ + +string CCodeGen::PTR_CONST() +{ + return "const "; +} + +string CCodeGen::PTR_CONST_END() +{ + return ""; +} + +std::ostream &CCodeGen::OPEN_ARRAY( string type, string name ) +{ + out << "#if defined(__GNUC__)\n"; + out << "static __attribute__((used)) const " << type << " " << name << "[] = {\n"; + out << "#else\n"; + out << "static const " << type << " " << name << "[] = {\n"; + out << "#endif\n"; + return out; +} + +std::ostream &CCodeGen::CLOSE_ARRAY() +{ + return out << "};\n"; +} + +std::ostream &CCodeGen::STATIC_VAR( string type, string name ) +{ + out << "enum {" << name; + return out; +} + +string CCodeGen::UINT( ) +{ + return "unsigned int"; +} + +string CCodeGen::ARR_OFF( string ptr, string offset ) +{ + return ptr + " + " + offset; +} + +string CCodeGen::CAST( string type ) +{ + return "(" + type + ")"; +} + +string CCodeGen::NULL_ITEM() +{ + return "0"; +} + +string CCodeGen::POINTER() +{ + return " *"; +} + +std::ostream &CCodeGen::SWITCH_DEFAULT() +{ + return out; +} + +string CCodeGen::CTRL_FLOW() +{ + return ""; +} + +void CCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + out << "#define " << DATA_PREFIX() << "ex_" << ex->name << " " << + KEY(ex->key) << "\n"; + } + out << "\n"; + } +} + +/* + * D Specific + */ + +string DCodeGen::NULL_ITEM() +{ + return "null"; +} + +string DCodeGen::POINTER() +{ + // multiple items seperated by commas can also be pointer types. + return "* "; +} + +string DCodeGen::PTR_CONST() +{ + return ""; +} + +string DCodeGen::PTR_CONST_END() +{ + return ""; +} + +std::ostream &DCodeGen::OPEN_ARRAY( string type, string name ) +{ + out << "static const " << type << "[] " << name << " = [\n"; + return out; +} + +std::ostream &DCodeGen::CLOSE_ARRAY() +{ + return out << "];\n"; +} + +std::ostream &DCodeGen::STATIC_VAR( string type, string name ) +{ + out << "static const " << type << " " << name; + return out; +} + +string DCodeGen::ARR_OFF( string ptr, string offset ) +{ + return "&" + ptr + "[" + offset + "]"; +} + +string DCodeGen::CAST( string type ) +{ + return "cast(" + type + ")"; +} + +string DCodeGen::UINT( ) +{ + return "uint"; +} + +std::ostream &DCodeGen::SWITCH_DEFAULT() +{ + out << " default: break;\n"; + return out; +} + +string DCodeGen::CTRL_FLOW() +{ + return "if (true) "; +} + +void DCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + out << "static const " << ALPH_TYPE() << " " << DATA_PREFIX() << + "ex_" << ex->name << " = " << KEY(ex->key) << ";\n"; + } + out << "\n"; + } +} + +/* + * End D-specific code. + */ + +/* + * D2 Specific + */ + +string D2CodeGen::NULL_ITEM() +{ + return "null"; +} + +string D2CodeGen::POINTER() +{ + // multiple items seperated by commas can also be pointer types. + return "* "; +} + +string D2CodeGen::PTR_CONST() +{ + return "const("; +} + +string D2CodeGen::PTR_CONST_END() +{ + return ")"; +} + +std::ostream &D2CodeGen::OPEN_ARRAY( string type, string name ) +{ + out << "enum " << type << "[] " << name << " = [\n"; + return out; +} + +std::ostream &D2CodeGen::CLOSE_ARRAY() +{ + return out << "];\n"; +} + +std::ostream &D2CodeGen::STATIC_VAR( string type, string name ) +{ + out << "enum " << type << " " << name; + return out; +} + +string D2CodeGen::ARR_OFF( string ptr, string offset ) +{ + return "&" + ptr + "[" + offset + "]"; +} + +string D2CodeGen::CAST( string type ) +{ + return "cast(" + type + ")"; +} + +string D2CodeGen::UINT( ) +{ + return "uint"; +} + +std::ostream &D2CodeGen::SWITCH_DEFAULT() +{ + out << " default: break;\n"; + return out; +} + +string D2CodeGen::CTRL_FLOW() +{ + return "if (true) "; +} + +void D2CodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + out << "enum " << ALPH_TYPE() << " " << DATA_PREFIX() << + "ex_" << ex->name << " = " << KEY(ex->key) << ";\n"; + } + out << "\n"; + } +} + +void D2CodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish, bool csForced ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << "{{"; + INLINE_LIST( ret, item->children, targState, inFinish, csForced ); + ret << "}}"; + } +} + +void D2CodeGen::ACTION( ostream &ret, GenAction *action, int targState, + bool inFinish, bool csForced ) +{ + /* Write the preprocessor line info for going into the source file. */ + cdLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + ret << "\t{{"; + INLINE_LIST( ret, action->inlineList, targState, inFinish, csForced ); + ret << "}}\n"; +} + +/* + * End D2-specific code. + */ + +void FsmCodeGen::finishRagelDef() +{ + if ( codeStyle == GenGoto || codeStyle == GenFGoto || + codeStyle == GenIpGoto || codeStyle == GenSplit ) + { + /* For directly executable machines there is no required state + * ordering. Choose a depth-first ordering to increase the + * potential for fall-throughs. */ + redFsm->depthFirstOrdering(); + } + else { + /* The frontend will do this for us, but it may be a good idea to + * force it if the intermediate file is edited. */ + redFsm->sortByStateId(); + } + + /* Choose default transitions and the single transition. */ + redFsm->chooseDefaultSpan(); + + /* Maybe do flat expand, otherwise choose single. */ + if ( codeStyle == GenFlat || codeStyle == GenFFlat ) + redFsm->makeFlat(); + else + redFsm->chooseSingle(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + if ( codeStyle == GenSplit ) + redFsm->partitionFsm( numSplitPartitions ); + + if ( codeStyle == GenIpGoto || codeStyle == GenSplit ) + redFsm->setInTrans(); + + /* Anlayze Machine will find the final action reference counts, among + * other things. We will use these in reporting the usage + * of fsm directives in action code. */ + analyzeMachine(); + + /* Determine if we should use indicies. */ + calcIndexSize(); +} + +ostream &FsmCodeGen::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &FsmCodeGen::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + diff --git a/contrib/tools/ragel6/cdcodegen.h b/contrib/tools/ragel6/cdcodegen.h new file mode 100644 index 0000000000..b9a84906d5 --- /dev/null +++ b/contrib/tools/ragel6/cdcodegen.h @@ -0,0 +1,258 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDCODEGEN_H +#define _CDCODEGEN_H + +#include <iostream> +#include <string> +#include <stdio.h> +#include "common.h" +#include "gendata.h" + +using std::string; +using std::ostream; + +/* Integer array line length. */ +#define IALL 8 + +/* Forwards. */ +struct RedFsmAp; +struct RedStateAp; +struct CodeGenData; +struct GenAction; +struct NameInst; +struct GenInlineItem; +struct GenInlineList; +struct RedAction; +struct LongestMatch; +struct LongestMatchPart; + +string itoa( int i ); + +/* + * class FsmCodeGen + */ +class FsmCodeGen : public CodeGenData +{ +public: + FsmCodeGen( ostream &out ); + virtual ~FsmCodeGen() {} + + virtual void finishRagelDef(); + virtual void writeInit(); + virtual void writeStart(); + virtual void writeFirstFinal(); + virtual void writeError(); + +protected: + string FSM_NAME(); + string START_STATE_ID(); + ostream &ACTIONS_ARRAY(); + string GET_WIDE_KEY(); + string GET_WIDE_KEY( RedStateAp *state ); + string TABS( int level ); + string KEY( Key key ); + string WIDE_KEY( RedStateAp *state, Key key ); + string LDIR_PATH( char *path ); + virtual void ACTION( ostream &ret, GenAction *action, int targState, + bool inFinish, bool csForced ); + void CONDITION( ostream &ret, GenAction *condition ); + string ALPH_TYPE(); + string WIDE_ALPH_TYPE(); + string ARRAY_TYPE( unsigned long maxVal ); + + bool isAlphTypeSigned(); + bool isWideAlphTypeSigned(); + + virtual string ARR_OFF( string ptr, string offset ) = 0; + virtual string CAST( string type ) = 0; + virtual string UINT() = 0; + virtual string NULL_ITEM() = 0; + virtual string POINTER() = 0; + virtual string GET_KEY(); + virtual ostream &SWITCH_DEFAULT() = 0; + + string P(); + string PE(); + string vEOF(); + + string ACCESS(); + string vCS(); + string STACK(); + string TOP(); + string TOKSTART(); + string TOKEND(); + string ACT(); + + string DATA_PREFIX(); + string PM() { return "_" + DATA_PREFIX() + "partition_map"; } + string C() { return "_" + DATA_PREFIX() + "cond_spaces"; } + string CK() { return "_" + DATA_PREFIX() + "cond_keys"; } + string K() { return "_" + DATA_PREFIX() + "trans_keys"; } + string I() { return "_" + DATA_PREFIX() + "indicies"; } + string CO() { return "_" + DATA_PREFIX() + "cond_offsets"; } + string KO() { return "_" + DATA_PREFIX() + "key_offsets"; } + string IO() { return "_" + DATA_PREFIX() + "index_offsets"; } + string CL() { return "_" + DATA_PREFIX() + "cond_lengths"; } + string SL() { return "_" + DATA_PREFIX() + "single_lengths"; } + string RL() { return "_" + DATA_PREFIX() + "range_lengths"; } + string A() { return "_" + DATA_PREFIX() + "actions"; } + string TA() { return "_" + DATA_PREFIX() + "trans_actions"; } + string TT() { return "_" + DATA_PREFIX() + "trans_targs"; } + string TSA() { return "_" + DATA_PREFIX() + "to_state_actions"; } + string FSA() { return "_" + DATA_PREFIX() + "from_state_actions"; } + string EA() { return "_" + DATA_PREFIX() + "eof_actions"; } + string ET() { return "_" + DATA_PREFIX() + "eof_trans"; } + string SP() { return "_" + DATA_PREFIX() + "key_spans"; } + string CSP() { return "_" + DATA_PREFIX() + "cond_key_spans"; } + string START() { return DATA_PREFIX() + "start"; } + string ERROR() { return DATA_PREFIX() + "error"; } + string FIRST_FINAL() { return DATA_PREFIX() + "first_final"; } + string CTXDATA() { return DATA_PREFIX() + "ctxdata"; } + + void EOF_CHECK( ostream &ret ); + + void INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish, bool csForced ); + virtual void GOTO( ostream &ret, int gotoDest, bool inFinish ) = 0; + virtual void CALL( ostream &ret, int callDest, int targState, bool inFinish ) = 0; + virtual void NEXT( ostream &ret, int nextDest, bool inFinish ) = 0; + virtual void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, + int targState, bool inFinish ) = 0; + virtual void RET( ostream &ret, bool inFinish ) = 0; + virtual void BREAK( ostream &ret, int targState, bool csForced ) = 0; + virtual void CURS( ostream &ret, bool inFinish ) = 0; + virtual void TARGS( ostream &ret, bool inFinish, int targState ) = 0; + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, + int inFinish, bool csForced ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + virtual void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish, bool csForced ); + void STATE_IDS(); + + string ERROR_STATE(); + string FIRST_FINAL_STATE(); + + virtual string PTR_CONST() = 0; + virtual string PTR_CONST_END() = 0; + virtual ostream &OPEN_ARRAY( string type, string name ) = 0; + virtual ostream &CLOSE_ARRAY() = 0; + virtual ostream &STATIC_VAR( string type, string name ) = 0; + + virtual string CTRL_FLOW() = 0; + + ostream &source_warning(const InputLoc &loc); + ostream &source_error(const InputLoc &loc); + + unsigned int arrayTypeSize( unsigned long maxVal ); + + bool outLabelUsed; + bool testEofUsed; + bool againLabelUsed; + bool useIndicies; + + void genLineDirective( ostream &out ); + +public: + /* Determine if we should use indicies. */ + virtual void calcIndexSize() {} +}; + +class CCodeGen : virtual public FsmCodeGen +{ +public: + CCodeGen( ostream &out ) : FsmCodeGen(out) {} + + virtual string NULL_ITEM(); + virtual string POINTER(); + virtual ostream &SWITCH_DEFAULT(); + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + virtual string ARR_OFF( string ptr, string offset ); + virtual string CAST( string type ); + virtual string UINT(); + virtual string PTR_CONST(); + virtual string PTR_CONST_END(); + virtual string CTRL_FLOW(); + + virtual void writeExports(); +}; + +class DCodeGen : virtual public FsmCodeGen +{ +public: + DCodeGen( ostream &out ) : FsmCodeGen(out) {} + + virtual string NULL_ITEM(); + virtual string POINTER(); + virtual ostream &SWITCH_DEFAULT(); + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + virtual string ARR_OFF( string ptr, string offset ); + virtual string CAST( string type ); + virtual string UINT(); + virtual string PTR_CONST(); + virtual string PTR_CONST_END(); + virtual string CTRL_FLOW(); + + virtual void writeExports(); +}; + +class D2CodeGen : virtual public FsmCodeGen +{ +public: + D2CodeGen( ostream &out ) : FsmCodeGen(out) {} + + virtual string NULL_ITEM(); + virtual string POINTER(); + virtual ostream &SWITCH_DEFAULT(); + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + virtual string ARR_OFF( string ptr, string offset ); + virtual string CAST( string type ); + virtual string UINT(); + virtual string PTR_CONST(); + virtual string PTR_CONST_END(); + virtual string CTRL_FLOW(); + + virtual void writeExports(); + virtual void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish, bool csForced ); + virtual void ACTION( ostream &ret, GenAction *action, int targState, + bool inFinish, bool csForced ); + +}; + +#endif diff --git a/contrib/tools/ragel6/cdfflat.cpp b/contrib/tools/ragel6/cdfflat.cpp new file mode 100644 index 0000000000..79cc5be3cb --- /dev/null +++ b/contrib/tools/ragel6/cdfflat.cpp @@ -0,0 +1,384 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdfflat.h" +#include "redfsm.h" +#include "gendata.h" + +std::ostream &FFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &FFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &FFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + +/* Write out the function for a transition. */ +std::ostream &FFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &FFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FFlatCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void FFlatCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void FFlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {\n" + " int _slen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << ";\n"; + out << " int _trans"; + + if ( redFsm->anyConditions() ) + out << ", _cond"; + + out << ";\n"; + + out << + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_keys;\n" + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxIndex) << PTR_CONST_END() << POINTER() << "_inds;\n"; + + if ( redFsm->anyConditions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxCond) << PTR_CONST_END() << POINTER() << "_conds;\n" + " " << WIDE_ALPH_TYPE() << " _widec;\n"; + } + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " switch ( " << TA() << "[_trans] ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/contrib/tools/ragel6/cdfflat.h b/contrib/tools/ragel6/cdfflat.h new file mode 100644 index 0000000000..e5cbfbd9e9 --- /dev/null +++ b/contrib/tools/ragel6/cdfflat.h @@ -0,0 +1,85 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDFFLAT_H +#define _CDFFLAT_H + +#include <iostream> +#include "cdflat.h" + +/* Forwards. */ +struct CodeGenData; + +/* + * FFlatCodeGen + */ +class FFlatCodeGen : public FlatCodeGen +{ +protected: + FFlatCodeGen( ostream &out ) : FsmCodeGen(out), FlatCodeGen(out) {} + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +/* + * CFFlatCodeGen + */ +struct CFFlatCodeGen + : public FFlatCodeGen, public CCodeGen +{ + CFFlatCodeGen( ostream &out ) : + FsmCodeGen(out), FFlatCodeGen(out), CCodeGen(out) {} +}; + +/* + * DFFlatCodeGen + */ +struct DFFlatCodeGen + : public FFlatCodeGen, public DCodeGen +{ + DFFlatCodeGen( ostream &out ) : + FsmCodeGen(out), FFlatCodeGen(out), DCodeGen(out) {} +}; + +/* + * D2FFlatCodeGen + */ +struct D2FFlatCodeGen + : public FFlatCodeGen, public D2CodeGen +{ + D2FFlatCodeGen( ostream &out ) : + FsmCodeGen(out), FFlatCodeGen(out), D2CodeGen(out) {} +}; + +#endif diff --git a/contrib/tools/ragel6/cdfgoto.cpp b/contrib/tools/ragel6/cdfgoto.cpp new file mode 100644 index 0000000000..3b229deab0 --- /dev/null +++ b/contrib/tools/ragel6/cdfgoto.cpp @@ -0,0 +1,296 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdfgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +std::ostream &FGotoCodeGen::EXEC_ACTIONS() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* We are at the start of a glob, write the case. */ + out << "f" << redAct->actListId << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tgoto _again;\n"; + } + } + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FGotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FGotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &FGotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &FGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\tcase " << st->id << ": "; + + /* Jump to the func. */ + out << "goto f" << st->eofAction->actListId << ";\n"; + } + } + + return out; +} + +unsigned int FGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + return act; +} + +unsigned int FGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + return act; +} + +unsigned int FGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + return act; +} + +void FGotoCodeGen::writeData() +{ + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void FGotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + out << + " switch ( " << vCS() << " ) {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_ACTIONS() << "\n"; + + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " switch ( " << vCS() << " ) {\n"; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + SWITCH_DEFAULT() << + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/contrib/tools/ragel6/cdfgoto.h b/contrib/tools/ragel6/cdfgoto.h new file mode 100644 index 0000000000..27e980e9ea --- /dev/null +++ b/contrib/tools/ragel6/cdfgoto.h @@ -0,0 +1,85 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDFGOTO_H +#define _CDFGOTO_H + +#include <iostream> +#include "cdgoto.h" + +/* Forwards. */ +struct CodeGenData; + + +/* + * class FGotoCodeGen + */ +class FGotoCodeGen : public GotoCodeGen +{ +public: + FGotoCodeGen( ostream &out ) : FsmCodeGen(out), GotoCodeGen(out) {} + + std::ostream &EXEC_ACTIONS(); + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &FINISH_CASES(); + std::ostream &EOF_ACTION_SWITCH(); + unsigned int TO_STATE_ACTION( RedStateAp *state ); + unsigned int FROM_STATE_ACTION( RedStateAp *state ); + unsigned int EOF_ACTION( RedStateAp *state ); + + virtual void writeData(); + virtual void writeExec(); +}; + +/* + * class CFGotoCodeGen + */ +struct CFGotoCodeGen + : public FGotoCodeGen, public CCodeGen +{ + CFGotoCodeGen( ostream &out ) : + FsmCodeGen(out), FGotoCodeGen(out), CCodeGen(out) {} +}; + +/* + * class DFGotoCodeGen + */ +struct DFGotoCodeGen + : public FGotoCodeGen, public DCodeGen +{ + DFGotoCodeGen( ostream &out ) : + FsmCodeGen(out), FGotoCodeGen(out), DCodeGen(out) {} +}; + +/* + * class DFGotoCodeGen + */ +struct D2FGotoCodeGen + : public FGotoCodeGen, public D2CodeGen +{ + D2FGotoCodeGen( ostream &out ) : + FsmCodeGen(out), FGotoCodeGen(out), D2CodeGen(out) {} +}; + +#endif diff --git a/contrib/tools/ragel6/cdflat.cpp b/contrib/tools/ragel6/cdflat.cpp new file mode 100644 index 0000000000..b28a95a660 --- /dev/null +++ b/contrib/tools/ragel6/cdflat.cpp @@ -0,0 +1,887 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdflat.h" +#include "redfsm.h" +#include "gendata.h" + +std::ostream &FlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &FlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &FlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + +std::ostream &FlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &FlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &FlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &FlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &FlatCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &FlatCodeGen::FLAT_INDEX_OFFSET() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + if ( st->transList != 0 ) + curIndOffset += keyOps->span( st->lowKey, st->highKey ); + + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::KEY_SPANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->transList != 0 ) + span = keyOps->span( st->lowKey, st->highKey ); + out << span; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::TO_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::FROM_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::EOF_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::EOF_TRANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &FlatCodeGen::COND_KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just cond low key and cond high key. */ + out << KEY( st->condLowKey ) << ", "; + out << KEY( st->condHighKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &FlatCodeGen::COND_KEY_SPANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->condList != 0 ) + span = keyOps->span( st->condLowKey, st->condHighKey ); + out << span; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &FlatCodeGen::CONDS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->condList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + if ( st->condList[pos] != 0 ) + out << st->condList[pos]->condSpaceId + 1 << ", "; + else + out << "0, "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &FlatCodeGen::COND_INDEX_OFFSET() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + if ( st->condList != 0 ) + curIndOffset += keyOps->span( st->condLowKey, st->condHighKey ); + } + out << "\n"; + return out; +} + + +std::ostream &FlatCodeGen::KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just low key and high key. */ + out << KEY( st->lowKey ) << ", "; + out << KEY( st->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &FlatCodeGen::INDICIES() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->transList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + out << st->transList[pos]->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) + out << st->defTrans->id << ", "; + + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &FlatCodeGen::TRANS_TARGS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Save the position. Needed for eofTargs. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id; + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalStates % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + + +std::ostream &FlatCodeGen::TRANS_ACTIONS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalAct % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + +void FlatCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << ARR_OFF( K(), "(" + vCS() + "<<1)" ) << ";\n" + " _inds = " << ARR_OFF( I(), IO() + "[" + vCS() + "]" ) << ";\n" + "\n" + " _slen = " << SP() << "[" << vCS() << "];\n" + " _trans = _inds[ _slen > 0 && _keys[0] <=" << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= _keys[1] ?\n" + " " << GET_WIDE_KEY() << " - _keys[0] : _slen ];\n" + "\n"; +} + +void FlatCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{"; + + ret << vCS() << " = " << gotoDest << ";"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void FlatCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{"; + + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << "); "; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void FlatCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void FlatCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void FlatCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void FlatCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; +} + +void FlatCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{"; + + ret << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << callDest << ";"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + + +void FlatCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish, false ); + ret << ");"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + ret << "}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + + +void FlatCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false, false ); + ret << "}"; + } + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void FlatCodeGen::BREAK( ostream &ret, int targState, bool csForced ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void FlatCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void FlatCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n"; + + out << + " _keys = " << ARR_OFF( CK(), "(" + vCS() + "<<1)" ) << ";\n" + " _conds = " << ARR_OFF( C(), CO() + "[" + vCS() + "]" ) << ";\n" + "\n" + " _slen = " << CSP() << "[" << vCS() << "];\n" + " _cond = _slen > 0 && _keys[0] <=" << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= _keys[1] ?\n" + " _conds[" << GET_WIDE_KEY() << " - _keys[0]] : 0;\n" + "\n"; + + out << + " switch ( _cond ) {\n"; + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId + 1 << ": {\n"; + out << TABS(2) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << " }\n"; + out << " break;\n"; + } + + SWITCH_DEFAULT(); + + out << + " }\n"; +} + +void FlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {\n" + " int _slen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " int _trans"; + + if ( redFsm->anyConditions() ) + out << ", _cond"; + out << ";\n"; + + if ( redFsm->anyToStateActions() || + redFsm->anyRegActions() || redFsm->anyFromStateActions() ) + { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << PTR_CONST_END() << POINTER() << "_acts;\n" + " " << UINT() << " _nacts;\n"; + } + + out << + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_keys;\n" + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxIndex) << PTR_CONST_END() << POINTER() << "_inds;\n"; + + if ( redFsm->anyConditions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxCond) << PTR_CONST_END() << POINTER() << "_conds;\n" + " " << WIDE_ALPH_TYPE() << " _widec;\n"; + } + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), FSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " _acts = " << ARR_OFF( A(), TA() + "[_trans]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *(_acts++) )\n {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), TSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << PTR_CONST_END() << + POINTER() << "__acts = " << + ARR_OFF( A(), EA() + "[" + vCS() + "]" ) << ";\n" + " " << UINT() << " __nacts = " << CAST(UINT()) << " *__acts++;\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( *__acts++ ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/contrib/tools/ragel6/cdflat.h b/contrib/tools/ragel6/cdflat.h new file mode 100644 index 0000000000..96c65cb1eb --- /dev/null +++ b/contrib/tools/ragel6/cdflat.h @@ -0,0 +1,118 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDFLAT_H +#define _CDFLAT_H + +#include <iostream> +#include "cdcodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; + +/* + * FlatCodeGen + */ +class FlatCodeGen : virtual public FsmCodeGen +{ +public: + FlatCodeGen( ostream &out ) : FsmCodeGen(out) {} + virtual ~FlatCodeGen() { } + +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &FLAT_INDEX_OFFSET(); + std::ostream &KEY_SPANS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + void LOCATE_TRANS(); + + std::ostream &COND_INDEX_OFFSET(); + void COND_TRANSLATE(); + std::ostream &CONDS(); + std::ostream &COND_KEYS(); + std::ostream &COND_KEY_SPANS(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState, bool csForced ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +/* + * CFlatCodeGen + */ +struct CFlatCodeGen + : public FlatCodeGen, public CCodeGen +{ + CFlatCodeGen( ostream &out ) : + FsmCodeGen(out), FlatCodeGen(out), CCodeGen(out) {} +}; + +/* + * DFlatCodeGen + */ +struct DFlatCodeGen + : public FlatCodeGen, public DCodeGen +{ + DFlatCodeGen( ostream &out ) : + FsmCodeGen(out), FlatCodeGen(out), DCodeGen(out) {} +}; + +/* + * D2FlatCodeGen + */ +struct D2FlatCodeGen + : public FlatCodeGen, public D2CodeGen +{ + D2FlatCodeGen( ostream &out ) : + FsmCodeGen(out), FlatCodeGen(out), D2CodeGen(out) {} +}; + +#endif diff --git a/contrib/tools/ragel6/cdftable.cpp b/contrib/tools/ragel6/cdftable.cpp new file mode 100644 index 0000000000..f5584e5d97 --- /dev/null +++ b/contrib/tools/ragel6/cdftable.cpp @@ -0,0 +1,436 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdftable.h" +#include "redfsm.h" +#include "gendata.h" + +/* Determine if we should use indicies or not. */ +void FTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActListId) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActListId) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &FTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &FTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &FTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + + +/* Write out the function for a transition. */ +std::ostream &FTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &FTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &FTabCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void FTabCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void FTabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {\n" + " int _klen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_keys;\n" + " int _trans;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + out << "_match:\n"; + + if ( useIndicies ) + out << " _trans = " << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " switch ( " << TA() << "[_trans] ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/contrib/tools/ragel6/cdftable.h b/contrib/tools/ragel6/cdftable.h new file mode 100644 index 0000000000..4a8264ce75 --- /dev/null +++ b/contrib/tools/ragel6/cdftable.h @@ -0,0 +1,87 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDFTABLE_H +#define _CDFTABLE_H + +#include <iostream> +#include "cdtable.h" + +/* Forwards. */ +struct CodeGenData; + + +/* + * FTabCodeG\verb|e + */ +class FTabCodeGen : public TabCodeGen +{ +protected: + FTabCodeGen( ostream &out ) : FsmCodeGen(out), TabCodeGen(out) {} + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void writeData(); + virtual void writeExec(); + virtual void calcIndexSize(); +}; + + +/* + * CFTabCodeGen + */ +struct CFTabCodeGen + : public FTabCodeGen, public CCodeGen +{ + CFTabCodeGen( ostream &out ) : + FsmCodeGen(out), FTabCodeGen(out), CCodeGen(out) {} +}; + +/* + * class DFTabCodeGen + */ +struct DFTabCodeGen + : public FTabCodeGen, public DCodeGen +{ + DFTabCodeGen( ostream &out ) : + FsmCodeGen(out), FTabCodeGen(out), DCodeGen(out) {} +}; + +/* + * class D2FTabCodeGen + */ +struct D2FTabCodeGen + : public FTabCodeGen, public D2CodeGen +{ + D2FTabCodeGen( ostream &out ) : + FsmCodeGen(out), FTabCodeGen(out), D2CodeGen(out) {} +}; + +#endif diff --git a/contrib/tools/ragel6/cdgoto.cpp b/contrib/tools/ragel6/cdgoto.cpp new file mode 100644 index 0000000000..86b3d23fef --- /dev/null +++ b/contrib/tools/ragel6/cdgoto.cpp @@ -0,0 +1,839 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdgoto.h" +#include "redfsm.h" +#include "bstmap.h" +#include "gendata.h" + +/* Emit the goto to take for a given transition. */ +std::ostream &GotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + out << TABS(level) << "goto tr" << trans->id << ";"; + return out; +} + +std::ostream &GotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GotoCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void GotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + /* Label the state. */ + out << "case " << state->id << ":\n"; +} + + +void GotoCodeGen::emitSingleSwitch( RedStateAp *state ) +{ + /* Load up the singles. */ + int numSingles = state->outSingle.length(); + RedTransEl *data = state->outSingle.data; + + if ( numSingles == 1 ) { + /* If there is a single single key then write it out as an if. */ + out << "\tif ( " << GET_WIDE_KEY(state) << " == " << + WIDE_KEY(state, data[0].lowKey) << " )\n\t\t"; + + /* Virtual function for writing the target of the transition. */ + TRANS_GOTO(data[0].value, 0) << "\n"; + } + else if ( numSingles > 1 ) { + /* Write out single keys in a switch if there is more than one. */ + out << "\tswitch( " << GET_WIDE_KEY(state) << " ) {\n"; + + /* Write out the single indicies. */ + for ( int j = 0; j < numSingles; j++ ) { + out << "\t\tcase " << WIDE_KEY(state, data[j].lowKey) << ": "; + TRANS_GOTO(data[j].value, 0) << "\n"; + } + + /* Emits a default case for D code. */ + SWITCH_DEFAULT(); + + /* Close off the transition switch. */ + out << "\t}\n"; + } +} + +void GotoCodeGen::emitRangeBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + RedTransEl *data = state->outRange.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid].lowKey == keyOps->minKey; + bool limitHigh = data[mid].highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " < " << + WIDE_KEY(state, data[mid].lowKey) << " ) {\n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " > " << + WIDE_KEY(state, data[mid].highKey) << " ) {\n"; + emitRangeBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " < " << + WIDE_KEY(state, data[mid].lowKey) << " ) {\n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " <= " << + WIDE_KEY(state, data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " > " << + WIDE_KEY(state, data[mid].highKey) << " ) {\n"; + emitRangeBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " >= " << + WIDE_KEY(state, data[mid].lowKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << WIDE_KEY(state, data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " && " << GET_WIDE_KEY(state) << " <= " << + WIDE_KEY(state, data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " <= " << + WIDE_KEY(state, data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if ( " << WIDE_KEY(state, data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } +} + +void GotoCodeGen::STATE_GOTO_ERROR() +{ + /* Label the state and bail immediately. */ + outLabelUsed = true; + RedStateAp *state = redFsm->errState; + out << "case " << state->id << ":\n"; + out << " goto _out;\n"; +} + +void GotoCodeGen::COND_TRANSLATE( GenStateCond *stateCond, int level ) +{ + GenCondSpace *condSpace = stateCond->condSpace; + out << TABS(level) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(level) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } +} + +void GotoCodeGen::emitCondBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + GenStateCond **data = state->stateCondVect.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid]->lowKey == keyOps->minKey; + bool limitHigh = data[mid]->highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if ( " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " ) {\n"; + emitCondBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "} else if ( " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " ) {\n"; + emitCondBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if ( " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " ) {\n"; + emitCondBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if ( " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " ) {\n"; + emitCondBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_KEY() << " >= " << + KEY(data[mid]->lowKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " && " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " )\n {"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + COND_TRANSLATE(data[mid], level); + } + } +} + +std::ostream &GotoCodeGen::STATE_GOTOS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* Writing code above state gotos. */ + GOTO_HEADER( st ); + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + } + } + return out; +} + +std::ostream &GotoCodeGen::TRANSITIONS() +{ + /* Emit any transitions that have functions and that go to + * this state. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Write the label for the transition so it can be jumped to. */ + out << " tr" << trans->id << ": "; + + /* Destination state. */ + if ( trans->action != 0 && trans->action->anyCurStateRef() ) + out << "_ps = " << vCS() << ";"; + out << vCS() << " = " << trans->targ->id << "; "; + + if ( trans->action != 0 ) { + /* Write out the transition func. */ + out << "goto f" << trans->action->actListId << ";\n"; + } + else { + /* No code to execute, just loop around. */ + out << "goto _again;\n"; + } + } + return out; +} + +std::ostream &GotoCodeGen::EXEC_FUNCS() +{ + /* Make labels that set acts and jump to execFuncs. Loop func indicies. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + out << " f" << redAct->actListId << ": " << + "_acts = " << ARR_OFF(A(), itoa( redAct->location+1 ) ) << ";" + " goto execFuncs;\n"; + } + } + + out << + "\n" + "execFuncs:\n" + " _nacts = *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + " goto _again;\n"; + return out; +} + +unsigned int GotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +unsigned int GotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +unsigned int GotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + +std::ostream &GotoCodeGen::TO_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = TO_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &GotoCodeGen::FROM_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = FROM_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &GotoCodeGen::EOF_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = EOF_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &GotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\tcase " << st->id << ": "; + + /* Write the goto func. */ + out << "goto f" << st->eofAction->actListId << ";\n"; + } + } + + return out; +} + +void GotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{"; + + ret << vCS() << " = " << gotoDest << ";"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void GotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{"; + + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void GotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void GotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void GotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void GotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; +} + +void GotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{"; + + ret << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << callDest << ";"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void GotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{"; + + ret << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish, false ); + ret << ");"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void GotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false, false ); + ret << "}"; + } + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + ret << "}"; +} + +void GotoCodeGen::BREAK( ostream &ret, int targState, bool csForced ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void GotoCodeGen::writeData() +{ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void GotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << PTR_CONST_END() << POINTER() << "_acts;\n" + " " << UINT() << " _nacts;\n"; + } + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), FSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + out << + " switch ( " << vCS() << " ) {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_FUNCS() << "\n"; + + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), TSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " switch ( " << vCS() << " ) {\n"; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + SWITCH_DEFAULT() << + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << PTR_CONST_END() << + POINTER() << "__acts = " << + ARR_OFF( A(), EA() + "[" + vCS() + "]" ) << ";\n" + " " << UINT() << " __nacts = " << CAST(UINT()) << " *__acts++;\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( *__acts++ ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/contrib/tools/ragel6/cdgoto.h b/contrib/tools/ragel6/cdgoto.h new file mode 100644 index 0000000000..e0869c8d9d --- /dev/null +++ b/contrib/tools/ragel6/cdgoto.h @@ -0,0 +1,119 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDGOTO_H +#define _CDGOTO_H + +#include <iostream> +#include "cdcodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; +struct GenStateCond; + +/* + * Goto driven fsm. + */ +class GotoCodeGen : virtual public FsmCodeGen +{ +public: + GotoCodeGen( ostream &out ) : FsmCodeGen(out) {} + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &STATE_GOTOS(); + std::ostream &TRANSITIONS(); + std::ostream &EXEC_FUNCS(); + std::ostream &FINISH_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState, bool csForced ); + + virtual unsigned int TO_STATE_ACTION( RedStateAp *state ); + virtual unsigned int FROM_STATE_ACTION( RedStateAp *state ); + virtual unsigned int EOF_ACTION( RedStateAp *state ); + + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + + void COND_TRANSLATE( GenStateCond *stateCond, int level ); + void emitCondBSearch( RedStateAp *state, int level, int low, int high ); + void STATE_CONDS( RedStateAp *state, bool genDefault ); + + virtual std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + + void emitSingleSwitch( RedStateAp *state ); + void emitRangeBSearch( RedStateAp *state, int level, int low, int high ); + + /* Called from STATE_GOTOS just before writing the gotos */ + virtual void GOTO_HEADER( RedStateAp *state ); + virtual void STATE_GOTO_ERROR(); + + virtual void writeData(); + virtual void writeExec(); +}; + +/* + * class CGotoCodeGen + */ +struct CGotoCodeGen + : public GotoCodeGen, public CCodeGen +{ + CGotoCodeGen( ostream &out ) : + FsmCodeGen(out), GotoCodeGen(out), CCodeGen(out) {} +}; + +/* + * class DGotoCodeGen + */ +struct DGotoCodeGen + : public GotoCodeGen, public DCodeGen +{ + DGotoCodeGen( ostream &out ) : + FsmCodeGen(out), GotoCodeGen(out), DCodeGen(out) {} +}; + +/* + * class D2GotoCodeGen + */ +struct D2GotoCodeGen + : public GotoCodeGen, public D2CodeGen +{ + D2GotoCodeGen( ostream &out ) : + FsmCodeGen(out), GotoCodeGen(out), D2CodeGen(out) {} +}; + +#endif diff --git a/contrib/tools/ragel6/cdipgoto.cpp b/contrib/tools/ragel6/cdipgoto.cpp new file mode 100644 index 0000000000..7735f631b6 --- /dev/null +++ b/contrib/tools/ragel6/cdipgoto.cpp @@ -0,0 +1,495 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdipgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +bool IpGotoCodeGen::useAgainLabel() +{ + return redFsm->anyActionRets() || + redFsm->anyActionByValControl() || + redFsm->anyRegNextStmt(); +} + +void IpGotoCodeGen::EOF_CHECK( ostream &ret, int gotoDest ) +{ + ret << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof" << gotoDest << ";\n"; + + testEofUsed = true; +} + +void IpGotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret, gotoDest ); + + ret << CTRL_FLOW() << "goto st" << gotoDest << ";"; + + ret << "}"; +} + +void IpGotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << targState << ";"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret, callDest ); + + ret << CTRL_FLOW() << "goto st" << callDest << ";"; + + ret << "}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void IpGotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{"; + + ret << STACK() << "[" << TOP() << "++] = " << targState << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; + + if ( inFinish && !noEnd ) + FsmCodeGen::EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void IpGotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false, false ); + ret << "}"; + } + + if ( inFinish && !noEnd ) + FsmCodeGen::EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void IpGotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{"; + + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; + + if ( inFinish && !noEnd ) + FsmCodeGen::EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + ret << "}"; +} + +void IpGotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void IpGotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; +} + +void IpGotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void IpGotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << targState; +} + +void IpGotoCodeGen::BREAK( ostream &ret, int targState, bool csForced ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; "; + if ( !csForced ) + ret << vCS() << " = " << targState << "; "; + ret << CTRL_FLOW() << "goto _out;}"; +} + +bool IpGotoCodeGen::IN_TRANS_ACTIONS( RedStateAp *state ) +{ + bool anyWritten = false; + + /* Emit any transitions that have actions and that go to this state. */ + for ( int it = 0; it < state->numInTrans; it++ ) { + RedTransAp *trans = state->inTrans[it]; + if ( trans->action != 0 && trans->labelNeeded ) { + /* Remember that we wrote an action so we know to write the + * line directive for going back to the output. */ + anyWritten = true; + + /* Write the label for the transition so it can be jumped to. */ + out << "tr" << trans->id << ":\n"; + + /* If the action contains a next, then we must preload the current + * state since the action may or may not set it. */ + if ( trans->action->anyNextStmt() ) + out << " " << vCS() << " = " << trans->targ->id << ";\n"; + + /* Write each action in the list. */ + for ( GenActionTable::Iter item = trans->action->key; item.lte(); item++ ) { + ACTION( out, item->value, trans->targ->id, false, + trans->action->anyNextStmt() ); + } + + /* If the action contains a next then we need to reload, otherwise + * jump directly to the target state. */ + if ( trans->action->anyNextStmt() ) + out << "\tgoto _again;\n"; + else + out << "\tgoto st" << trans->targ->id << ";\n"; + } + } + + return anyWritten; +} + +/* Called from GotoCodeGen::STATE_GOTOS just before writing the gotos for each + * state. */ +void IpGotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + bool anyWritten = IN_TRANS_ACTIONS( state ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + if ( state->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->toStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, state->id, false, + state->toStateAction->anyNextStmt() ); + } + } + + /* Advance and test buffer pos. */ + if ( state->labelNeeded ) { + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _test_eof" << state->id << ";\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + } + + /* Give the state a switch case. */ + out << "case " << state->id << ":\n"; + + if ( state->fromStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->fromStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, state->id, false, + state->fromStateAction->anyNextStmt() ); + } + } + + if ( anyWritten ) + genLineDirective( out ); + + /* Record the prev state if necessary. */ + if ( state->anyRegCurStateRef() ) + out << " _ps = " << state->id << ";\n"; +} + +void IpGotoCodeGen::STATE_GOTO_ERROR() +{ + /* In the error state we need to emit some stuff that usually goes into + * the header. */ + RedStateAp *state = redFsm->errState; + bool anyWritten = IN_TRANS_ACTIONS( state ); + + /* No case label needed since we don't switch on the error state. */ + if ( anyWritten ) + genLineDirective( out ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + /* Break out here. */ + outLabelUsed = true; + out << vCS() << " = " << state->id << ";\n"; + out << " goto _out;\n"; +} + + +/* Emit the goto to take for a given transition. */ +std::ostream &IpGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto tr" << trans->id << ";"; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto st" << trans->targ->id << ";"; + } + return out; +} + +std::ostream &IpGotoCodeGen::EXIT_STATES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->outNeeded ) { + testEofUsed = true; + out << " _test_eof" << st->id << ": " << vCS() << " = " << + st->id << "; goto _test_eof;\n"; + } + } + return out; +} + +std::ostream &IpGotoCodeGen::AGAIN_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + out << + " case " << st->id << ": goto st" << st->id << ";\n"; + } + return out; +} + +std::ostream &IpGotoCodeGen::FINISH_CASES() +{ + bool anyWritten = false; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofAction != 0 ) { + if ( st->eofAction->eofRefs == 0 ) + st->eofAction->eofRefs = new IntSet; + st->eofAction->eofRefs->insert( st->id ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + if ( act->eofRefs != 0 ) { + for ( IntSet::Iter pst = *act->eofRefs; pst.lte(); pst++ ) + out << " case " << *pst << ": \n"; + + /* Remember that we wrote a trans so we know to write the + * line directive for going back to the output. */ + anyWritten = true; + + /* Write each action in the eof action list. */ + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) + ACTION( out, item->value, STATE_ERR_STATE, true, false ); + out << "\tbreak;\n"; + } + } + + if ( anyWritten ) + genLineDirective( out ); + return out; +} + +void IpGotoCodeGen::setLabelsNeeded( GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: { + /* Mark the target as needing a label. */ + item->targState->labelNeeded = true; + break; + } + default: break; + } + + if ( item->children != 0 ) + setLabelsNeeded( item->children ); + } +} + +/* Set up labelNeeded flag for each state. */ +void IpGotoCodeGen::setLabelsNeeded() +{ + /* If we use the _again label, then we the _again switch, which uses all + * labels. */ + if ( useAgainLabel() ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = true; + } + else { + /* Do not use all labels by default, init all labelNeeded vars to false. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = false; + + /* Walk all transitions and set only those that have targs. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* If there is no action with a next statement, then the label will be + * needed. */ + if ( trans->action == 0 || !trans->action->anyNextStmt() ) + trans->targ->labelNeeded = true; + + /* Need labels for states that have goto or calls in action code + * invoked on characters (ie, not from out action code). */ + if ( trans->action != 0 ) { + /* Loop the actions. */ + for ( GenActionTable::Iter act = trans->action->key; act.lte(); act++ ) { + /* Get the action and walk it's tree. */ + setLabelsNeeded( act->value->inlineList ); + } + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofAction != 0 ) { + for ( GenActionTable::Iter item = st->eofAction->key; item.lte(); item++ ) + setLabelsNeeded( item->value->inlineList ); + } + } + } + + if ( !noEnd ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st != redFsm->errState ) + st->outNeeded = st->labelNeeded; + } + } +} + +void IpGotoCodeGen::writeData() +{ + STATE_IDS(); +} + +void IpGotoCodeGen::writeExec() +{ + /* Must set labels immediately before writing because we may depend on the + * noend write option. */ + setLabelsNeeded(); + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( useAgainLabel() ) { + out << + " goto _resume;\n" + "\n" + "_again:\n" + " switch ( " << vCS() << " ) {\n"; + AGAIN_CASES() << + " default: break;\n" + " }\n" + "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << "_resume:\n"; + } + + out << + " switch ( " << vCS() << " )\n {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n"; + EXIT_STATES() << + "\n"; + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n" + " switch ( " << vCS() << " ) {\n"; + FINISH_CASES(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << + " }\n"; +} diff --git a/contrib/tools/ragel6/cdipgoto.h b/contrib/tools/ragel6/cdipgoto.h new file mode 100644 index 0000000000..4e92926e25 --- /dev/null +++ b/contrib/tools/ragel6/cdipgoto.h @@ -0,0 +1,107 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDIPGOTO_H +#define _CDIPGOTO_H + +#include <iostream> +#include "cdgoto.h" + +/* Forwards. */ +struct CodeGenData; + +/* + * class FGotoCodeGen + */ +class IpGotoCodeGen : public GotoCodeGen +{ +public: + IpGotoCodeGen( ostream &out ) : FsmCodeGen(out), GotoCodeGen(out) {} + + std::ostream &EXIT_STATES(); + std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + std::ostream &FINISH_CASES(); + std::ostream &AGAIN_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &ret, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void BREAK( ostream &ret, int targState, bool csForced ); + + virtual void writeData(); + virtual void writeExec(); + +protected: + bool useAgainLabel(); + + void EOF_CHECK( ostream &ret, int gotoDest ); + + /* Called from GotoCodeGen::STATE_GOTOS just before writing the gotos for + * each state. */ + bool IN_TRANS_ACTIONS( RedStateAp *state ); + void GOTO_HEADER( RedStateAp *state ); + void STATE_GOTO_ERROR(); + + /* Set up labelNeeded flag for each state. */ + void setLabelsNeeded( GenInlineList *inlineList ); + void setLabelsNeeded(); +}; + + +/* + * class CIpGotoCodeGen + */ +struct CIpGotoCodeGen + : public IpGotoCodeGen, public CCodeGen +{ + CIpGotoCodeGen( ostream &out ) : + FsmCodeGen(out), IpGotoCodeGen(out), CCodeGen(out) {} +}; + +/* + * class DIpGotoCodeGen + */ +struct DIpGotoCodeGen + : public IpGotoCodeGen, public DCodeGen +{ + DIpGotoCodeGen( ostream &out ) : + FsmCodeGen(out), IpGotoCodeGen(out), DCodeGen(out) {} +}; + +/* + * class D2IpGotoCodeGen + */ +struct D2IpGotoCodeGen + : public IpGotoCodeGen, public D2CodeGen +{ + D2IpGotoCodeGen( ostream &out ) : + FsmCodeGen(out), IpGotoCodeGen(out), D2CodeGen(out) {} +}; + +#endif diff --git a/contrib/tools/ragel6/cdsplit.cpp b/contrib/tools/ragel6/cdsplit.cpp new file mode 100644 index 0000000000..f494b730f5 --- /dev/null +++ b/contrib/tools/ragel6/cdsplit.cpp @@ -0,0 +1,526 @@ +/* + * Copyright 2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + + +#include "ragel.h" +#include "cdsplit.h" +#include "gendata.h" +#include <assert.h> + +using std::ostream; +using std::ios; +using std::endl; + +/* Emit the goto to take for a given transition. */ +std::ostream &SplitCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + if ( trans->targ->partition == currentPartition ) { + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto tr" << trans->id << ";"; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto st" << trans->targ->id << ";"; + } + } + else { + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto ptr" << trans->id << ";"; + trans->partitionBoundary = true; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto pst" << trans->targ->id << ";"; + trans->targ->partitionBoundary = true; + } + } + return out; +} + +/* Called from before writing the gotos for each state. */ +void SplitCodeGen::GOTO_HEADER( RedStateAp *state, bool stateInPartition ) +{ + bool anyWritten = IN_TRANS_ACTIONS( state ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + if ( state->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->toStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, state->id, false, + state->toStateAction->anyNextStmt() ); + } + } + + /* Advance and test buffer pos. */ + if ( state->labelNeeded ) { + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out" << state->id << ";\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + } + + /* Give the state a switch case. */ + out << "case " << state->id << ":\n"; + + if ( state->fromStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->fromStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, state->id, false, + state->fromStateAction->anyNextStmt() ); + } + } + + if ( anyWritten ) + genLineDirective( out ); + + /* Record the prev state if necessary. */ + if ( state->anyRegCurStateRef() ) + out << " _ps = " << state->id << ";\n"; +} + +std::ostream &SplitCodeGen::STATE_GOTOS( int partition ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partition == partition ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* We call into the base of the goto which calls back into us + * using virtual functions. Set the current partition rather + * than coding parameter passing throughout. */ + currentPartition = partition; + + /* Writing code above state gotos. */ + GOTO_HEADER( st, st->partition == partition ); + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + } + } + } + return out; +} + + +std::ostream &SplitCodeGen::PART_TRANS( int partition ) +{ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + if ( trans->partitionBoundary ) { + out << + "ptr" << trans->id << ":\n"; + + if ( trans->action != 0 ) { + /* If the action contains a next, then we must preload the current + * state since the action may or may not set it. */ + if ( trans->action->anyNextStmt() ) + out << " " << vCS() << " = " << trans->targ->id << ";\n"; + + /* Write each action in the list. */ + for ( GenActionTable::Iter item = trans->action->key; item.lte(); item++ ) { + ACTION( out, item->value, trans->targ->id, false, + trans->action->anyNextStmt() ); + } + } + + out << + " goto pst" << trans->targ->id << ";\n"; + trans->targ->partitionBoundary = true; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partitionBoundary ) { + out << + " pst" << st->id << ":\n" + " " << vCS() << " = " << st->id << ";\n"; + + if ( st->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + for ( GenActionTable::Iter item = st->toStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, st->id, false, + st->toStateAction->anyNextStmt() ); + } + genLineDirective( out ); + } + + ptOutLabelUsed = true; + out << " goto _pt_out; \n"; + } + } + return out; +} + +std::ostream &SplitCodeGen::EXIT_STATES( int partition ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partition == partition && st->outNeeded ) { + outLabelUsed = true; + out << " _out" << st->id << ": " << vCS() << " = " << + st->id << "; goto _out; \n"; + } + } + return out; +} + + +std::ostream &SplitCodeGen::PARTITION( int partition ) +{ + outLabelUsed = false; + ptOutLabelUsed = false; + + /* Initialize the partition boundaries, which get set during the writing + * of states. After the state writing we will */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + trans->partitionBoundary = false; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->partitionBoundary = false; + + out << " " << ALPH_TYPE() << " *p = *_pp, *pe = *_ppe;\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( useAgainLabel() ) { + out << + " goto _resume;\n" + "\n" + "_again:\n" + " switch ( " << vCS() << " ) {\n"; + AGAIN_CASES() << + " default: break;\n" + " }\n" + "\n"; + + + if ( !noEnd ) { + outLabelUsed = true; + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out;\n"; + + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << + "_resume:\n"; + } + + out << + " switch ( " << vCS() << " )\n {\n"; + STATE_GOTOS( partition ); + SWITCH_DEFAULT() << + " }\n"; + PART_TRANS( partition ); + EXIT_STATES( partition ); + + if ( outLabelUsed ) { + out << + "\n" + " _out:\n" + " *_pp = p;\n" + " *_ppe = pe;\n" + " return 0;\n"; + } + + if ( ptOutLabelUsed ) { + out << + "\n" + " _pt_out:\n" + " *_pp = p;\n" + " *_ppe = pe;\n" + " return 1;\n"; + } + + return out; +} + +std::ostream &SplitCodeGen::PART_MAP() +{ + int *partMap = new int[redFsm->stateList.length()]; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + partMap[st->id] = st->partition; + + out << "\t"; + int totalItem = 0; + for ( int i = 0; i < redFsm->stateList.length(); i++ ) { + out << partMap[i]; + if ( i != redFsm->stateList.length() - 1 ) { + out << ", "; + if ( ++totalItem % IALL == 0 ) + out << "\n\t"; + } + } + + delete[] partMap; + return out; +} + +void SplitCodeGen::writeData() +{ + out << + "static const int " << START() << " = " << START_STATE_ID() << ";\n" + "\n"; + + if ( !noFinal ) { + out << + "static const int " << FIRST_FINAL() << " = " << FIRST_FINAL_STATE() << ";\n" + "\n"; + } + + if ( !noError ) { + out << + "static const int " << ERROR() << " = " << ERROR_STATE() << ";\n" + "\n"; + } + + + OPEN_ARRAY( ARRAY_TYPE(numSplitPartitions), PM() ); + PART_MAP(); + CLOSE_ARRAY() << + "\n"; + + for ( int p = 0; p < redFsm->nParts; p++ ) { + out << "int partition" << p << "( " << ALPH_TYPE() << " **_pp, " << ALPH_TYPE() << + " **_ppe, struct " << FSM_NAME() << " *fsm );\n"; + } + out << "\n"; +} + +std::ostream &SplitCodeGen::ALL_PARTITIONS() +{ + /* compute the format string. */ + int width = 0, high = redFsm->nParts - 1; + while ( high > 0 ) { + width++; + high /= 10; + } + assert( width <= 8 ); + char suffFormat[] = "_%6.6d.c"; + suffFormat[2] = suffFormat[4] = ( '0' + width ); + + for ( int p = 0; p < redFsm->nParts; p++ ) { + char suffix[10]; + sprintf( suffix, suffFormat, p ); + const char *fn = fileNameFromStem( sourceFileName, suffix ); + const char *include = fileNameFromStem( sourceFileName, ".h" ); + + /* Create the filter on the output and open it. */ + output_filter *partFilter = new output_filter( fn ); + partFilter->open( fn, ios::out|ios::trunc ); + if ( !partFilter->is_open() ) { + error() << "error opening " << fn << " for writing" << endl; + exit(1); + } + + /* Attach the new file to the output stream. */ + std::streambuf *prev_rdbuf = out.rdbuf( partFilter ); + + out << + "#include \"" << include << "\"\n" + "int partition" << p << "( " << ALPH_TYPE() << " **_pp, " << ALPH_TYPE() << + " **_ppe, struct " << FSM_NAME() << " *fsm )\n" + "{\n"; + PARTITION( p ) << + "}\n\n"; + out.flush(); + + /* Fix the output stream. */ + out.rdbuf( prev_rdbuf ); + } + return out; +} + + +void SplitCodeGen::writeExec() +{ + /* Must set labels immediately before writing because we may depend on the + * noend write option. */ + setLabelsNeeded(); + out << + " {\n" + " int _stat = 0;\n"; + + if ( !noEnd ) { + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _out;\n"; + } + + out << " goto _resume;\n"; + + /* In this reentry, to-state actions have already been executed on the + * partition-switch exit from the last partition. */ + out << "_reenter:\n"; + + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out;\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << "_resume:\n"; + + out << + " switch ( " << PM() << "[" << vCS() << "] ) {\n"; + for ( int p = 0; p < redFsm->nParts; p++ ) { + out << + " case " << p << ":\n" + " _stat = partition" << p << "( &p, &pe, fsm );\n" + " break;\n"; + } + out << + " }\n" + " if ( _stat )\n" + " goto _reenter;\n"; + + if ( !noEnd ) + out << " _out: {}\n"; + + out << + " }\n"; + + ALL_PARTITIONS(); +} + +void SplitCodeGen::setLabelsNeeded( RedStateAp *fromState, GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: { + /* In split code gen we only need labels for transitions across + * partitions. */ + if ( fromState->partition == item->targState->partition ){ + /* Mark the target as needing a label. */ + item->targState->labelNeeded = true; + } + break; + } + default: break; + } + + if ( item->children != 0 ) + setLabelsNeeded( fromState, item->children ); + } +} + +void SplitCodeGen::setLabelsNeeded( RedStateAp *fromState, RedTransAp *trans ) +{ + /* In the split code gen we don't need labels for transitions across + * partitions. */ + if ( fromState->partition == trans->targ->partition ) { + /* If there is no action with a next statement, then the label will be + * needed. */ + trans->labelNeeded = true; + if ( trans->action == 0 || !trans->action->anyNextStmt() ) + trans->targ->labelNeeded = true; + } + + /* Need labels for states that have goto or calls in action code + * invoked on characters (ie, not from out action code). */ + if ( trans->action != 0 ) { + /* Loop the actions. */ + for ( GenActionTable::Iter act = trans->action->key; act.lte(); act++ ) { + /* Get the action and walk it's tree. */ + setLabelsNeeded( fromState, act->value->inlineList ); + } + } +} + +/* Set up labelNeeded flag for each state. */ +void SplitCodeGen::setLabelsNeeded() +{ + /* If we use the _again label, then we the _again switch, which uses all + * labels. */ + if ( useAgainLabel() ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = true; + } + else { + /* Do not use all labels by default, init all labelNeeded vars to false. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = false; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + trans->labelNeeded = false; + + /* Walk all transitions and set only those that have targs. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + for ( RedTransList::Iter tel = st->outRange; tel.lte(); tel++ ) + setLabelsNeeded( st, tel->value ); + + for ( RedTransList::Iter tel = st->outSingle; tel.lte(); tel++ ) + setLabelsNeeded( st, tel->value ); + + if ( st->defTrans != 0 ) + setLabelsNeeded( st, st->defTrans ); + } + } + + if ( !noEnd ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->outNeeded = st->labelNeeded; + } + else { + if ( redFsm->errState != 0 ) + redFsm->errState->outNeeded = true; + + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Any state with a transition in that has a break will need an + * out label. */ + if ( trans->action != 0 && trans->action->anyBreakStmt() ) + trans->targ->outNeeded = true; + } + } +} + diff --git a/contrib/tools/ragel6/cdsplit.h b/contrib/tools/ragel6/cdsplit.h new file mode 100644 index 0000000000..03dea78b50 --- /dev/null +++ b/contrib/tools/ragel6/cdsplit.h @@ -0,0 +1,80 @@ +/* + * Copyright 2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDSPLIT_H +#define _CDSPLIT_H + +#include "cdipgoto.h" + +class SplitCodeGen : public IpGotoCodeGen +{ +public: + SplitCodeGen( ostream &out ) : FsmCodeGen(out), IpGotoCodeGen(out) {} + + bool ptOutLabelUsed; + + std::ostream &PART_MAP(); + std::ostream &EXIT_STATES( int partition ); + std::ostream &PART_TRANS( int partition ); + std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + void GOTO_HEADER( RedStateAp *state, bool stateInPartition ); + std::ostream &STATE_GOTOS( int partition ); + std::ostream &PARTITION( int partition ); + std::ostream &ALL_PARTITIONS(); + void writeData(); + void writeExec(); + void writeParts(); + + void setLabelsNeeded( RedStateAp *fromState, GenInlineList *inlineList ); + void setLabelsNeeded( RedStateAp *fromState, RedTransAp *trans ); + void setLabelsNeeded(); + + int currentPartition; +}; + +struct CSplitCodeGen + : public SplitCodeGen, public CCodeGen +{ + CSplitCodeGen( ostream &out ) : + FsmCodeGen(out), SplitCodeGen(out), CCodeGen(out) {} +}; + +/* + * class DIpGotoCodeGen + */ +struct DSplitCodeGen + : public SplitCodeGen, public DCodeGen +{ + DSplitCodeGen( ostream &out ) : + FsmCodeGen(out), SplitCodeGen(out), DCodeGen(out) {} +}; + +/* + * class D2SplitCodeGen + */ +struct D2SplitCodeGen + : public SplitCodeGen, public D2CodeGen +{ + D2SplitCodeGen( ostream &out ) : + FsmCodeGen(out), SplitCodeGen(out), D2CodeGen(out) {} +}; + +#endif diff --git a/contrib/tools/ragel6/cdtable.cpp b/contrib/tools/ragel6/cdtable.cpp new file mode 100644 index 0000000000..6bcb79ea84 --- /dev/null +++ b/contrib/tools/ragel6/cdtable.cpp @@ -0,0 +1,1132 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cdtable.h" +#include "redfsm.h" +#include "gendata.h" + +/* Determine if we should use indicies or not. */ +void TabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActionLoc) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActionLoc) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &TabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &TabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &TabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + + +std::ostream &TabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &TabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &TabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &TabCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &TabCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &TabCodeGen::COND_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->stateCondList.length(); + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::KEY_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->outSingle.length() + st->outRange.length()*2; + } + out << "\n"; + return out; +} + + +std::ostream &TabCodeGen::INDEX_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + curIndOffset += st->outSingle.length() + st->outRange.length(); + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::COND_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->stateCondList.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &TabCodeGen::SINGLE_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->outSingle.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::RANGE_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit length of range index. */ + out << st->outRange.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::TO_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::FROM_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::EOF_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &TabCodeGen::EOF_TRANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &TabCodeGen::COND_KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Lower key. */ + out << KEY( sc->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + /* Upper key. */ + out << KEY( sc->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &TabCodeGen::COND_SPACES() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Cond Space id. */ + out << sc->condSpace->condSpaceId << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &TabCodeGen::KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << KEY( stel->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Loop the state's transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + /* Lower key. */ + out << KEY( rtel->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + /* Upper key. */ + out << KEY( rtel->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &TabCodeGen::INDICIES() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << stel->value->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + out << rtel->value->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + out << st->defTrans->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &TabCodeGen::TRANS_TARGS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default target state. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Add any eof transitions that have not yet been written out above. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + trans->pos = totalTrans; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + + +std::ostream &TabCodeGen::TRANS_ACTIONS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Add any eof transitions that have not yet been written out above. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &TabCodeGen::TRANS_TARGS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Record the position, need this for eofTrans. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id; + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalStates % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + + +std::ostream &TabCodeGen::TRANS_ACTIONS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalAct % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + +void TabCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << ARR_OFF( K(), KO() + "[" + vCS() + "]" ) << ";\n" + " _trans = " << IO() << "[" << vCS() << "];\n" + "\n" + " _klen = " << SL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_lower = _keys;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_mid;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_upper = _keys + _klen - 1;\n" + " while (1) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + ((_upper-_lower) >> 1);\n" + " if ( " << GET_WIDE_KEY() << " < *_mid )\n" + " _upper = _mid - 1;\n" + " else if ( " << GET_WIDE_KEY() << " > *_mid )\n" + " _lower = _mid + 1;\n" + " else {\n" + " _trans += " << CAST(UINT()) << "(_mid - _keys);\n" + " goto _match;\n" + " }\n" + " }\n" + " _keys += _klen;\n" + " _trans += _klen;\n" + " }\n" + "\n" + " _klen = " << RL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_lower = _keys;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_mid;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_upper = _keys + (_klen<<1) - 2;\n" + " while (1) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1);\n" + " if ( " << GET_WIDE_KEY() << " < _mid[0] )\n" + " _upper = _mid - 2;\n" + " else if ( " << GET_WIDE_KEY() << " > _mid[1] )\n" + " _lower = _mid + 2;\n" + " else {\n" + " _trans += " << CAST(UINT()) << "((_mid - _keys)>>1);\n" + " goto _match;\n" + " }\n" + " }\n" + " _trans += _klen;\n" + " }\n" + "\n"; +} + +void TabCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{"; + + ret << vCS() << " = " << gotoDest << ";"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void TabCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{"; + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << "); "; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void TabCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void TabCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void TabCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void TabCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; +} + +void TabCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << callDest << ";"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void TabCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{"; + + ret << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish, false ); + ret << ");"; + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void TabCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << + TOP() << "]; "; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false, false ); + ret << "}"; + } + + if ( inFinish && !noEnd ) + EOF_CHECK( ret ); + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void TabCodeGen::BREAK( ostream &ret, int targState, bool csForced ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void TabCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void TabCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n" + " _klen = " << CL() << "[" << vCS() << "];\n" + " _keys = " << ARR_OFF( CK(), "(" + CO() + "[" + vCS() + "]*2)" ) << ";\n" + " if ( _klen > 0 ) {\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_lower = _keys;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_mid;\n" + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_upper = _keys + (_klen<<1) - 2;\n" + " while (1) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1);\n" + " if ( " << GET_WIDE_KEY() << " < _mid[0] )\n" + " _upper = _mid - 2;\n" + " else if ( " << GET_WIDE_KEY() << " > _mid[1] )\n" + " _lower = _mid + 2;\n" + " else {\n" + " switch ( " << C() << "[" << CO() << "[" << vCS() << "]" + " + ((_mid - _keys)>>1)] ) {\n"; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId << ": {\n"; + out << TABS(2) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << + " break;\n" + " }\n"; + } + + SWITCH_DEFAULT(); + + out << + " }\n" + " break;\n" + " }\n" + " }\n" + " }\n" + "\n"; +} + +void TabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {\n" + " int _klen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " " << UINT() << " _trans;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << PTR_CONST_END() << + POINTER() << "_acts;\n" + " " << UINT() << " _nacts;\n"; + } + + out << + " " << PTR_CONST() << WIDE_ALPH_TYPE() << PTR_CONST_END() << POINTER() << "_keys;\n" + "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), FSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + out << "_match:\n"; + + if ( useIndicies ) + out << " _trans = " << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " _acts = " << ARR_OFF( A(), TA() + "[_trans]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 )\n {\n" + " switch ( *_acts++ )\n {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), TSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << CAST(UINT()) << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << PTR_CONST_END() << + POINTER() << "__acts = " << + ARR_OFF( A(), EA() + "[" + vCS() + "]" ) << ";\n" + " " << UINT() << " __nacts = " << CAST(UINT()) << " *__acts++;\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( *__acts++ ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/contrib/tools/ragel6/cdtable.h b/contrib/tools/ragel6/cdtable.h new file mode 100644 index 0000000000..b8df71b0cb --- /dev/null +++ b/contrib/tools/ragel6/cdtable.h @@ -0,0 +1,124 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CDTABLE_H +#define _CDTABLE_H + +#include <iostream> +#include "cdcodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; + +/* + * TabCodeGen + */ +class TabCodeGen : virtual public FsmCodeGen +{ +public: + TabCodeGen( ostream &out ) : FsmCodeGen(out) {} + virtual ~TabCodeGen() { } + virtual void writeData(); + virtual void writeExec(); + +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + std::ostream &COND_KEYS(); + std::ostream &COND_SPACES(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &COND_OFFSETS(); + std::ostream &KEY_OFFSETS(); + std::ostream &INDEX_OFFSETS(); + std::ostream &COND_LENS(); + std::ostream &SINGLE_LENS(); + std::ostream &RANGE_LENS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + std::ostream &TRANS_TARGS_WI(); + std::ostream &TRANS_ACTIONS_WI(); + void LOCATE_TRANS(); + + void COND_TRANSLATE(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState, bool csForced ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void calcIndexSize(); +}; + + +/* + * CTabCodeGen + */ +struct CTabCodeGen + : public TabCodeGen, public CCodeGen +{ + CTabCodeGen( ostream &out ) : + FsmCodeGen(out), TabCodeGen(out), CCodeGen(out) {} +}; + +/* + * DTabCodeGen + */ +struct DTabCodeGen + : public TabCodeGen, public DCodeGen +{ + DTabCodeGen( ostream &out ) : + FsmCodeGen(out), TabCodeGen(out), DCodeGen(out) {} +}; + +/* + * D2TabCodeGen + */ +struct D2TabCodeGen + : public TabCodeGen, public D2CodeGen +{ + D2TabCodeGen( ostream &out ) : + FsmCodeGen(out), TabCodeGen(out), D2CodeGen(out) {} +}; + +#endif diff --git a/contrib/tools/ragel6/common.cpp b/contrib/tools/ragel6/common.cpp new file mode 100644 index 0000000000..8e9f8ed02c --- /dev/null +++ b/contrib/tools/ragel6/common.cpp @@ -0,0 +1,380 @@ +/* + * Copyright 2006-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "pcheck.h" +#include "common.h" +#include "stdlib.h" +#include <string.h> +#include <assert.h> + +HostType hostTypesC[] = +{ + { "char", 0, "char", true, true, false, CHAR_MIN, CHAR_MAX, 0, 0, sizeof(char) }, + { "unsigned", "char", "uchar", false, true, false, 0, 0, 0, UCHAR_MAX, sizeof(unsigned char) }, + { "short", 0, "short", true, true, false, SHRT_MIN, SHRT_MAX, 0, 0, sizeof(short) }, + { "unsigned", "short", "ushort", false, true, false, 0, 0, 0, USHRT_MAX, sizeof(unsigned short) }, + { "int", 0, "int", true, true, false, INT_MIN, INT_MAX, 0, 0, sizeof(int) }, + { "unsigned", "int", "uint", false, true, false, 0, 0, 0, UINT_MAX, sizeof(unsigned int) }, + { "long", 0, "long", true, true, false, LONG_MIN, LONG_MAX, 0, 0, sizeof(long) }, + { "unsigned", "long", "ulong", false, true, false, 0, 0, 0, ULONG_MAX, sizeof(unsigned long) } +}; + +#define S8BIT_MIN -128 +#define S8BIT_MAX 127 + +#define U8BIT_MIN 0 +#define U8BIT_MAX 255 + +#define S16BIT_MIN -32768 +#define S16BIT_MAX 32767 + +#define U16BIT_MIN 0 +#define U16BIT_MAX 65535 + +#define S31BIT_MIN -1073741824L +#define S31BIT_MAX 1073741823L + +#define S32BIT_MIN -2147483648L +#define S32BIT_MAX 2147483647L + +#define U32BIT_MIN 0 +#define U32BIT_MAX 4294967295UL + +#define S64BIT_MIN -9223372036854775807LL +#define S64BIT_MAX 9223372036854775807LL + +#define U64BIT_MIN 0 +#define U64BIT_MAX 18446744073709551615ULL + +HostType hostTypesD[] = +{ + { "byte", 0, "byte", true, true, false, CHAR_MIN, CHAR_MAX, 0, 0, 1 }, + { "ubyte", 0, "ubyte", false, true, false, 0, 0, 0, UCHAR_MAX, 1 }, + { "char", 0, "char", false, true, false, 0, 0, 0, UCHAR_MAX, 1 }, + { "short", 0, "short", true, true, false, SHRT_MIN, SHRT_MAX, 0, 0, 2 }, + { "ushort", 0, "ushort", false, true, false, 0, 0, 0, USHRT_MAX, 2 }, + { "wchar", 0, "wchar", false, true, false, 0, 0, 0, USHRT_MAX, 2 }, + { "int", 0, "int", true, true, false, INT_MIN, INT_MAX, 0, 0, 4 }, + { "uint", 0, "uint", false, true, false, 0, 0, 0, UINT_MAX, 4 }, + { "dchar", 0, "dchar", false, true, false, 0, 0, 0, UINT_MAX, 4 } +}; + +HostType hostTypesGo[] = +{ + { "byte", 0, "uint8", false, true, false, 0, 0, U8BIT_MIN, U8BIT_MAX, 1 }, + { "int8", 0, "int8", true, true, false, S8BIT_MIN, S8BIT_MAX, 0, 0, 1 }, + { "uint8", 0, "uint8", false, true, false, 0, 0, U8BIT_MIN, U8BIT_MAX, 1 }, + { "int16", 0, "int16", true, true, false, S16BIT_MIN, S16BIT_MAX, 0, 0, 2 }, + { "uint16", 0, "uint16", false, true, false, 0, 0, U16BIT_MIN, U16BIT_MAX, 2 }, + { "int32", 0, "int32", true, true, false, S32BIT_MIN, S32BIT_MAX, 0, 0, 4 }, + { "uint32", 0, "uint32", false, true, false, 0, 0, U32BIT_MIN, U32BIT_MAX, 4 }, + { "int64", 0, "int64", true, true, false, S64BIT_MIN, S64BIT_MAX, 0, 0, 8 }, + { "uint64", 0, "uint64", false, true, false, 0, 0, U64BIT_MIN, U64BIT_MAX, 8 }, + { "rune", 0, "int32", true, true, true, S32BIT_MIN, S32BIT_MAX, 0, 0, 4 } +}; + +HostType hostTypesJava[] = +{ + { "byte", 0, "byte", true, true, false, CHAR_MIN, CHAR_MAX, 0, 0, 1 }, + { "short", 0, "short", true, true, false, SHRT_MIN, SHRT_MAX, 0, 0, 2 }, + { "char", 0, "char", false, true, false, 0, 0, 0, USHRT_MAX, 2 }, + { "int", 0, "int", true, true, false, INT_MIN, INT_MAX, 0, 0, 4 }, +}; + +/* What are the appropriate types for ruby? */ +HostType hostTypesRuby[] = +{ + { "char", 0, "char", true, true, false, CHAR_MIN, CHAR_MAX, 0, 0, 1 }, + { "int", 0, "int", true, true, false, INT_MIN, INT_MAX, 0, 0, 4 }, +}; + +HostType hostTypesCSharp[] = +{ + { "sbyte", 0, "sbyte", true, true, false, CHAR_MIN, CHAR_MAX, 0, 0, 1 }, + { "byte", 0, "byte", false, true, false, 0, 0, 0, UCHAR_MAX, 1 }, + { "short", 0, "short", true, true, false, SHRT_MIN, SHRT_MAX, 0, 0, 2 }, + { "ushort", 0, "ushort", false, true, false, 0, 0, 0, USHRT_MAX, 2 }, + { "char", 0, "char", false, true, true, 0, 0, 0, USHRT_MAX, 2 }, + { "int", 0, "int", true, true, false, INT_MIN, INT_MAX, 0, 0, 4 }, + { "uint", 0, "uint", false, true, false, 0, 0, 0, UINT_MAX, 4 }, + { "long", 0, "long", true, true, false, LONG_MIN, LONG_MAX, 0, 0, 8 }, + { "ulong", 0, "ulong", false, true, false, 0, 0, 0, ULONG_MAX, 8 } +}; + +HostType hostTypesOCaml[] = +{ + { "int", 0, "int", true, true, false, S31BIT_MIN, S31BIT_MAX, 0, 0, 4 }, +}; + +HostLang hostLangC = { HostLang::C, hostTypesC, 8, hostTypesC+0, true }; +HostLang hostLangD = { HostLang::D, hostTypesD, 9, hostTypesD+2, true }; +HostLang hostLangD2 = { HostLang::D2, hostTypesD, 9, hostTypesD+2, true }; +HostLang hostLangGo = { HostLang::Go, hostTypesGo, 10, hostTypesGo+0, false }; +HostLang hostLangJava = { HostLang::Java, hostTypesJava, 4, hostTypesJava+2, false }; +HostLang hostLangRuby = { HostLang::Ruby, hostTypesRuby, 2, hostTypesRuby+0, false }; +HostLang hostLangCSharp = { HostLang::CSharp, hostTypesCSharp, 9, hostTypesCSharp+4, true }; +HostLang hostLangOCaml = { HostLang::OCaml, hostTypesOCaml, 1, hostTypesOCaml+0, false }; + +HostLang *hostLang = &hostLangC; + +HostType *findAlphType( const char *s1 ) +{ + for ( int i = 0; i < hostLang->numHostTypes; i++ ) { + if ( strcmp( s1, hostLang->hostTypes[i].data1 ) == 0 && + hostLang->hostTypes[i].data2 == 0 ) + { + return hostLang->hostTypes + i; + } + } + + return 0; +} + +HostType *findAlphType( const char *s1, const char *s2 ) +{ + for ( int i = 0; i < hostLang->numHostTypes; i++ ) { + if ( strcmp( s1, hostLang->hostTypes[i].data1 ) == 0 && + hostLang->hostTypes[i].data2 != 0 && + strcmp( s2, hostLang->hostTypes[i].data2 ) == 0 ) + { + return hostLang->hostTypes + i; + } + } + + return 0; +} + +HostType *findAlphTypeInternal( const char *s1 ) +{ + for ( int i = 0; i < hostLang->numHostTypes; i++ ) { + if ( strcmp( s1, hostLang->hostTypes[i].internalName ) == 0 ) + return hostLang->hostTypes + i; + } + + return 0; +} + +/* Construct a new parameter checker with for paramSpec. */ +ParamCheck::ParamCheck( const char *paramSpec, int argc, const char **argv ) +: + state(noparam), + argOffset(0), + curArg(0), + iCurArg(1), + paramSpec(paramSpec), + argc(argc), + argv(argv) +{ +} + +/* Check a single option. Returns the index of the next parameter. Sets p to + * the arg character if valid, 0 otherwise. Sets parg to the parameter arg if + * there is one, NULL otherwise. */ +bool ParamCheck::check() +{ + bool requiresParam; + + if ( iCurArg >= argc ) { /* Off the end of the arg list. */ + state = noparam; + return false; + } + + if ( argOffset != 0 && *argOffset == 0 ) { + /* We are at the end of an arg string. */ + iCurArg += 1; + if ( iCurArg >= argc ) { + state = noparam; + return false; + } + argOffset = 0; + } + + if ( argOffset == 0 ) { + /* Set the current arg. */ + curArg = argv[iCurArg]; + + /* We are at the beginning of an arg string. */ + if ( argv[iCurArg] == 0 || /* Argv[iCurArg] is null. */ + argv[iCurArg][0] != '-' || /* Not a param. */ + argv[iCurArg][1] == 0 ) { /* Only a dash. */ + parameter = 0; + paramArg = 0; + + iCurArg += 1; + state = noparam; + return true; + } + argOffset = argv[iCurArg] + 1; + } + + /* Get the arg char. */ + char argChar = *argOffset; + + /* Loop over all the parms and look for a match. */ + const char *pSpec = paramSpec; + while ( *pSpec != 0 ) { + char pSpecChar = *pSpec; + + /* If there is a ':' following the char then + * it requires a parm. If a parm is required + * then move ahead two in the parmspec. Otherwise + * move ahead one in the parm spec. */ + if ( pSpec[1] == ':' ) { + requiresParam = true; + pSpec += 2; + } + else { + requiresParam = false; + pSpec += 1; + } + + /* Do we have a match. */ + if ( argChar == pSpecChar ) { + if ( requiresParam ) { + if ( argOffset[1] == 0 ) { + /* The param must follow. */ + if ( iCurArg + 1 == argc ) { + /* We are the last arg so there + * cannot be a parameter to it. */ + parameter = argChar; + paramArg = 0; + iCurArg += 1; + argOffset = 0; + state = invalid; + return true; + } + else { + /* the parameter to the arg is the next arg. */ + parameter = pSpecChar; + paramArg = argv[iCurArg + 1]; + iCurArg += 2; + argOffset = 0; + state = match; + return true; + } + } + else { + /* The param for the arg is built in. */ + parameter = pSpecChar; + paramArg = argOffset + 1; + iCurArg += 1; + argOffset = 0; + state = match; + return true; + } + } + else { + /* Good, we matched the parm and no + * arg is required. */ + parameter = pSpecChar; + paramArg = 0; + argOffset += 1; + state = match; + return true; + } + } + } + + /* We did not find a match. Bad Argument. */ + parameter = argChar; + paramArg = 0; + argOffset += 1; + state = invalid; + return true; +} + +/* Counts newlines before sending sync. */ +int output_filter::sync( ) +{ + line += 1; + return std::filebuf::sync(); +} + +/* Counts newlines before sending data out to file. */ +std::streamsize output_filter::xsputn( const char *s, std::streamsize n ) +{ + for ( int i = 0; i < n; i++ ) { + if ( s[i] == '\n' ) + line += 1; + } + return std::filebuf::xsputn( s, n ); +} + +/* Scans a string looking for the file extension. If there is a file + * extension then pointer returned points to inside the string + * passed in. Otherwise returns null. */ +const char *findFileExtension( const char *stemFile ) +{ + const char *ppos = stemFile + strlen(stemFile) - 1; + + /* Scan backwards from the end looking for the first dot. + * If we encounter a '/' before the first dot, then stop the scan. */ + while ( 1 ) { + /* If we found a dot or got to the beginning of the string then + * we are done. */ + if ( ppos == stemFile || *ppos == '.' ) + break; + + /* If we hit a / then there is no extension. Done. */ + if ( *ppos == '/' ) { + ppos = stemFile; + break; + } + ppos--; + } + + /* If we got to the front of the string then bail we + * did not find an extension */ + if ( ppos == stemFile ) + ppos = 0; + + return ppos; +} + +/* Make a file name from a stem. Removes the old filename suffix and + * replaces it with a new one. Returns a newed up string. */ +const char *fileNameFromStem( const char *stemFile, const char *suffix ) +{ + long len = strlen( stemFile ); + assert( len > 0 ); + + /* Get the extension. */ + const char *ppos = findFileExtension( stemFile ); + + /* If an extension was found, then shorten what we think the len is. */ + if ( ppos != 0 ) + len = ppos - stemFile; + + /* Make the return string from the stem and the suffix. */ + char *retVal = new char[ len + strlen( suffix ) + 1 ]; + strncpy( retVal, stemFile, len ); + strcpy( retVal + len, suffix ); + + return retVal; +} + +exit_object endp; + +void operator<<( std::ostream &out, exit_object & ) +{ + out << std::endl; + exit(1); +} diff --git a/contrib/tools/ragel6/common.h b/contrib/tools/ragel6/common.h new file mode 100644 index 0000000000..3ee71d26b0 --- /dev/null +++ b/contrib/tools/ragel6/common.h @@ -0,0 +1,392 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _COMMON_H +#define _COMMON_H + +#include <fstream> +#include <climits> +#include "dlist.h" + +/* Location in an input file. */ +struct InputLoc +{ + const char *fileName; + long line; + long col; +}; + + +typedef unsigned long long Size; + +struct Key +{ +private: + long key; + +public: + friend inline Key operator+(const Key key1, const Key key2); + friend inline Key operator-(const Key key1, const Key key2); + friend inline Key operator/(const Key key1, const Key key2); + friend inline long operator&(const Key key1, const Key key2); + + friend inline bool operator<( const Key key1, const Key key2 ); + friend inline bool operator<=( const Key key1, const Key key2 ); + friend inline bool operator>( const Key key1, const Key key2 ); + friend inline bool operator>=( const Key key1, const Key key2 ); + friend inline bool operator==( const Key key1, const Key key2 ); + friend inline bool operator!=( const Key key1, const Key key2 ); + + friend struct KeyOps; + + Key( ) {} + Key( const Key &key ) : key(key.key) {} + Key( long key ) : key(key) {} + + /* Returns the value used to represent the key. This value must be + * interpreted based on signedness. */ + long getVal() const { return key; }; + + /* Returns the key casted to a long long. This form of the key does not + * require any signedness interpretation. */ + long long getLongLong() const; + + /* Returns the distance from the key value to the maximum value that the + * key implementation can hold. */ + Size availableSpace() const; + + bool isUpper() const { return ( 'A' <= key && key <= 'Z' ); } + bool isLower() const { return ( 'a' <= key && key <= 'z' ); } + bool isPrintable() const + { + return ( 7 <= key && key <= 13 ) || ( 32 <= key && key < 127 ); + } + + Key toUpper() const + { return Key( 'A' + ( key - 'a' ) ); } + Key toLower() const + { return Key( 'a' + ( key - 'A' ) ); } + + void operator+=( const Key other ) + { + /* FIXME: must be made aware of isSigned. */ + key += other.key; + } + + void operator-=( const Key other ) + { + /* FIXME: must be made aware of isSigned. */ + key -= other.key; + } + + void operator|=( const Key other ) + { + /* FIXME: must be made aware of isSigned. */ + key |= other.key; + } + + /* Decrement. Needed only for ranges. */ + inline void decrement(); + inline void increment(); +}; + +struct HostType +{ + const char *data1; + const char *data2; + const char *internalName; + bool isSigned; + bool isOrd; + bool isChar; + long long sMinVal; + long long sMaxVal; + unsigned long long uMinVal; + unsigned long long uMaxVal; + unsigned int size; +}; + +struct HostLang +{ + /* Target language. */ + enum Lang + { + C, D, D2, Go, Java, Ruby, CSharp, OCaml + }; + + Lang lang; + HostType *hostTypes; + int numHostTypes; + HostType *defaultAlphType; + bool explicitUnsigned; +}; + +extern HostLang *hostLang; + +extern HostLang hostLangC; +extern HostLang hostLangD; +extern HostLang hostLangD2; +extern HostLang hostLangGo; +extern HostLang hostLangJava; +extern HostLang hostLangRuby; +extern HostLang hostLangCSharp; +extern HostLang hostLangOCaml; + +HostType *findAlphType( const char *s1 ); +HostType *findAlphType( const char *s1, const char *s2 ); +HostType *findAlphTypeInternal( const char *s1 ); + +/* An abstraction of the key operators that manages key operations such as + * comparison and increment according the signedness of the key. */ +struct KeyOps +{ + /* Default to signed alphabet. */ + KeyOps() : + isSigned(true), + alphType(0) + {} + + /* Default to signed alphabet. */ + KeyOps( bool isSigned ) + :isSigned(isSigned) {} + + bool isSigned; + Key minKey, maxKey; + HostType *alphType; + + void setAlphType( HostType *alphType ) + { + this->alphType = alphType; + isSigned = alphType->isSigned; + if ( isSigned ) { + minKey = (long) alphType->sMinVal; + maxKey = (long) alphType->sMaxVal; + } + else { + minKey = (long) (unsigned long) alphType->uMinVal; + maxKey = (long) (unsigned long) alphType->uMaxVal; + } + } + + /* Compute the distance between two keys. */ + Size span( Key key1, Key key2 ) + { + return isSigned ? + (unsigned long long)( + (long long)key2.key - + (long long)key1.key + 1) : + (unsigned long long)( + (unsigned long)key2.key) - + (unsigned long long)((unsigned long)key1.key) + 1; + } + + Size alphSize() + { return span( minKey, maxKey ); } + + HostType *typeSubsumes( long long maxVal ) + { + HostType *hostTypes = hostLang->hostTypes; + + for ( int i = 0; i < hostLang->numHostTypes; i++ ) { + long long typeMaxVal = hostTypes[i].isSigned ? hostTypes[i].sMaxVal : hostTypes[i].uMaxVal; + if ( maxVal <= typeMaxVal ) + return &hostLang->hostTypes[i]; + } + + return 0; + } + + HostType *typeSubsumes( bool isSigned, long long maxVal ) + { + HostType *hostTypes = hostLang->hostTypes; + + for ( int i = 0; i < hostLang->numHostTypes; i++ ) { + long long typeMaxVal = hostTypes[i].isSigned ? hostTypes[i].sMaxVal : hostTypes[i].uMaxVal; + if ( ( ( isSigned && hostTypes[i].isSigned ) || !isSigned ) && + maxVal <= typeMaxVal ) + return hostLang->hostTypes + i; + } + return 0; + } +}; + +extern KeyOps *keyOps; + +inline bool operator<( const Key key1, const Key key2 ) +{ + return keyOps->isSigned ? key1.key < key2.key : + (unsigned long)key1.key < (unsigned long)key2.key; +} + +inline bool operator<=( const Key key1, const Key key2 ) +{ + return keyOps->isSigned ? key1.key <= key2.key : + (unsigned long)key1.key <= (unsigned long)key2.key; +} + +inline bool operator>( const Key key1, const Key key2 ) +{ + return keyOps->isSigned ? key1.key > key2.key : + (unsigned long)key1.key > (unsigned long)key2.key; +} + +inline bool operator>=( const Key key1, const Key key2 ) +{ + return keyOps->isSigned ? key1.key >= key2.key : + (unsigned long)key1.key >= (unsigned long)key2.key; +} + +inline bool operator==( const Key key1, const Key key2 ) +{ + return key1.key == key2.key; +} + +inline bool operator!=( const Key key1, const Key key2 ) +{ + return key1.key != key2.key; +} + +/* Decrement. Needed only for ranges. */ +inline void Key::decrement() +{ + key = keyOps->isSigned ? key - 1 : ((unsigned long)key)-1; +} + +/* Increment. Needed only for ranges. */ +inline void Key::increment() +{ + key = keyOps->isSigned ? key+1 : ((unsigned long)key)+1; +} + +inline long long Key::getLongLong() const +{ + return keyOps->isSigned ? (long long)key : (long long)(unsigned long)key; +} + +inline Size Key::availableSpace() const +{ + if ( keyOps->isSigned ) + return (long long)LONG_MAX - (long long)key; + else + return (unsigned long long)ULONG_MAX - (unsigned long long)(unsigned long)key; +} + +inline Key operator+(const Key key1, const Key key2) +{ + /* FIXME: must be made aware of isSigned. */ + return Key( key1.key + key2.key ); +} + +inline Key operator-(const Key key1, const Key key2) +{ + /* FIXME: must be made aware of isSigned. */ + return Key( key1.key - key2.key ); +} + +inline long operator&(const Key key1, const Key key2) +{ + /* FIXME: must be made aware of isSigned. */ + return key1.key & key2.key; +} + +inline Key operator/(const Key key1, const Key key2) +{ + /* FIXME: must be made aware of isSigned. */ + return key1.key / key2.key; +} + +/* Filter on the output stream that keeps track of the number of lines + * output. */ +class output_filter : public std::filebuf +{ +public: + output_filter( const char *fileName ) : fileName(fileName), line(1) { } + + virtual int sync(); + virtual std::streamsize xsputn(const char* s, std::streamsize n); + + const char *fileName; + int line; +}; + +class cfilebuf : public std::streambuf +{ +public: + cfilebuf( char *fileName, FILE* file ) : fileName(fileName), file(file) { } + char *fileName; + FILE *file; + + int sync() + { + fflush( file ); + return 0; + } + + int overflow( int c ) + { + if ( c != EOF ) + fputc( c, file ); + return 0; + } + + std::streamsize xsputn( const char* s, std::streamsize n ) + { + std::streamsize written = fwrite( s, 1, n, file ); + return written; + } +}; + +class costream : public std::ostream +{ +public: + costream( cfilebuf *b ) : + std::ostream(b), b(b) {} + + ~costream() + { delete b; } + + void fclose() + { ::fclose( b->file ); } + + cfilebuf *b; +}; + + +const char *findFileExtension( const char *stemFile ); +const char *fileNameFromStem( const char *stemFile, const char *suffix ); + +struct Export +{ + Export( const char *name, Key key ) + : name(name), key(key) {} + + const char *name; + Key key; + + Export *prev, *next; +}; + +typedef DList<Export> ExportList; + +struct exit_object { }; +extern exit_object endp; +void operator<<( std::ostream &out, exit_object & ); + +#endif diff --git a/contrib/tools/ragel6/config.h b/contrib/tools/ragel6/config.h new file mode 100644 index 0000000000..8629850b31 --- /dev/null +++ b/contrib/tools/ragel6/config.h @@ -0,0 +1,26 @@ +/* ragel/config.h. Generated from config.h.in by configure. */ +/* ragel/config.h.in. Generated from configure.ac by autoheader. */ + +/* Name of package */ +#define PACKAGE "ragel" + +/* Define to the address where bug reports for this package should be sent. */ +#define PACKAGE_BUGREPORT "" + +/* Define to the full name of this package. */ +#define PACKAGE_NAME "ragel" + +/* Define to the full name and version of this package. */ +#define PACKAGE_STRING "ragel 6.10" + +/* Define to the one symbol short name of this package. */ +#define PACKAGE_TARNAME "ragel" + +/* Define to the home page for this package. */ +#define PACKAGE_URL "" + +/* Define to the version of this package. */ +#define PACKAGE_VERSION "6.10" + +/* Version number of package */ +#define VERSION "6.10" diff --git a/contrib/tools/ragel6/cscodegen.cpp b/contrib/tools/ragel6/cscodegen.cpp new file mode 100644 index 0000000000..37eb5211de --- /dev/null +++ b/contrib/tools/ragel6/cscodegen.cpp @@ -0,0 +1,824 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cscodegen.h" +#include "redfsm.h" +#include "gendata.h" +#include <sstream> +#include <iomanip> +#include <string> +#include <assert.h> + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; + +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + +void csharpLineDirective( ostream &out, const char *fileName, int line ) +{ + if ( noLineDirectives ) + out << "/* "; + + /* Write the preprocessor line info for to the input file. */ + out << "#line " << line << " \""; + for ( const char *pc = fileName; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + out << "\\\\"; + else + out << *pc; + } + out << '"'; + + if ( noLineDirectives ) + out << " */"; + + out << '\n'; +} + +void CSharpFsmCodeGen::genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + csharpLineDirective( out, filter->fileName, filter->line + 1 ); +} + + +/* Init code gen with in parameters. */ +CSharpFsmCodeGen::CSharpFsmCodeGen( ostream &out ) +: + CodeGenData(out) +{ +} + +unsigned int CSharpFsmCodeGen::arrayTypeSize( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + return arrayType->size; +} + +string CSharpFsmCodeGen::ARRAY_TYPE( unsigned long maxVal ) +{ + return ARRAY_TYPE( maxVal, false ); +} + +string CSharpFsmCodeGen::ARRAY_TYPE( unsigned long maxVal, bool forceSigned ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType; + if (forceSigned) + arrayType = keyOps->typeSubsumes(true, maxValLL); + else + arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + + string ret = arrayType->data1; + if ( arrayType->data2 != 0 ) { + ret += " "; + ret += arrayType->data2; + } + return ret; +} + +/* Write out the fsm name. */ +string CSharpFsmCodeGen::FSM_NAME() +{ + return fsmName; +} + +/* Emit the offset of the start state as a decimal integer. */ +string CSharpFsmCodeGen::START_STATE_ID() +{ + ostringstream ret; + ret << redFsm->startState->id; + return ret.str(); +}; + +/* Write out the array of actions. */ +std::ostream &CSharpFsmCodeGen::ACTIONS_ARRAY() +{ + out << "\t0, "; + int totalActions = 1; + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + /* Write out the length, which will never be the last character. */ + out << act->key.length() << ", "; + /* Put in a line break every 8 */ + if ( totalActions++ % 8 == 7 ) + out << "\n\t"; + + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) { + out << item->value->actionId; + if ( ! (act.last() && item.last()) ) + out << ", "; + + /* Put in a line break every 8 */ + if ( totalActions++ % 8 == 7 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +string CSharpFsmCodeGen::ACCESS() +{ + ostringstream ret; + if ( accessExpr != 0 ) + INLINE_LIST( ret, accessExpr, 0, false ); + return ret.str(); +} + + +string CSharpFsmCodeGen::P() +{ + ostringstream ret; + if ( pExpr == 0 ) + ret << "p"; + else { + ret << "("; + INLINE_LIST( ret, pExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::PE() +{ + ostringstream ret; + if ( peExpr == 0 ) + ret << "pe"; + else { + ret << "("; + INLINE_LIST( ret, peExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::vEOF() +{ + ostringstream ret; + if ( eofExpr == 0 ) + ret << "eof"; + else { + ret << "("; + INLINE_LIST( ret, eofExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::vCS() +{ + ostringstream ret; + if ( csExpr == 0 ) + ret << ACCESS() << "cs"; + else { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, csExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::TOP() +{ + ostringstream ret; + if ( topExpr == 0 ) + ret << ACCESS() + "top"; + else { + ret << "("; + INLINE_LIST( ret, topExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::STACK() +{ + ostringstream ret; + if ( stackExpr == 0 ) + ret << ACCESS() + "stack"; + else { + ret << "("; + INLINE_LIST( ret, stackExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::ACT() +{ + ostringstream ret; + if ( actExpr == 0 ) + ret << ACCESS() + "act"; + else { + ret << "("; + INLINE_LIST( ret, actExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::TOKSTART() +{ + ostringstream ret; + if ( tokstartExpr == 0 ) + ret << ACCESS() + "ts"; + else { + ret << "("; + INLINE_LIST( ret, tokstartExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::TOKEND() +{ + ostringstream ret; + if ( tokendExpr == 0 ) + ret << ACCESS() + "te"; + else { + ret << "("; + INLINE_LIST( ret, tokendExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string CSharpFsmCodeGen::GET_WIDE_KEY() +{ + if ( redFsm->anyConditions() ) + return "_widec"; + else + return GET_KEY(); +} + +string CSharpFsmCodeGen::GET_WIDE_KEY( RedStateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) + return "_widec"; + else + return GET_KEY(); +} + +string CSharpFsmCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + ret << "(*" << P() << ")"; + } + return ret.str(); +} + +/* Write out level number of tabs. Makes the nested binary search nice + * looking. */ +string CSharpFsmCodeGen::TABS( int level ) +{ + string result; + while ( level-- > 0 ) + result += "\t"; + return result; +} + +/* Write out a key from the fsm code gen. Depends on wether or not the key is + * signed. */ +string CSharpFsmCodeGen::KEY( Key key ) +{ + ostringstream ret; + if ( keyOps->isSigned || !hostLang->explicitUnsigned ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal() << 'u'; + return ret.str(); +} + +string CSharpFsmCodeGen::ALPHA_KEY( Key key ) +{ + ostringstream ret; + if (key.getVal() > 0xFFFF) { + ret << key.getVal(); + } else { + if ( keyOps->alphType->isChar ) + ret << "'\\u" << std::hex << std::setw(4) << std::setfill('0') << key.getVal() << "'"; + else + ret << key.getVal(); + } + //ret << "(char) " << key.getVal(); + return ret.str(); +} + +void CSharpFsmCodeGen::EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ) +{ + /* The parser gives fexec two children. The double brackets are for D + * code. If the inline list is a single word it will get interpreted as a + * C-style cast by the D compiler. */ + ret << "{" << P() << " = (("; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << "))-1;}"; +} + +void CSharpFsmCodeGen::LM_SWITCH( ostream &ret, GenInlineItem *item, + int targState, int inFinish ) +{ + ret << + " switch( " << ACT() << " ) {\n"; + + for ( GenInlineList::Iter lma = *item->children; lma.lte(); lma++ ) { + /* Write the case label, the action and the case break. */ + if ( lma->lmId < 0 ) + ret << " default:\n"; + else + ret << " case " << lma->lmId << ":\n"; + + /* Write the block and close it off. */ + ret << " {"; + INLINE_LIST( ret, lma->children, targState, inFinish ); + ret << "}\n"; + + ret << " break;\n"; + } + + ret << + " }\n" + "\t"; +} + +void CSharpFsmCodeGen::SET_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = " << item->lmId << ";"; +} + +void CSharpFsmCodeGen::SET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + /* The tokend action sets tokend. */ + ret << TOKEND() << " = " << P(); + if ( item->offset != 0 ) + out << "+" << item->offset; + out << ";"; +} + +void CSharpFsmCodeGen::GET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + ret << TOKEND(); +} + +void CSharpFsmCodeGen::INIT_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << NULL_ITEM() << ";"; +} + +void CSharpFsmCodeGen::INIT_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = 0;"; +} + +void CSharpFsmCodeGen::SET_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << P() << ";"; +} + +void CSharpFsmCodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << "{"; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << "}"; + } +} + + +/* Write out an inline tree structure. Walks the list and possibly calls out + * to virtual functions than handle language specific items in the tree. */ +void CSharpFsmCodeGen::INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Text: + ret << item->data; + break; + case GenInlineItem::Goto: + GOTO( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Call: + CALL( ret, item->targState->id, targState, inFinish ); + break; + case GenInlineItem::Next: + NEXT( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Ret: + RET( ret, inFinish ); + break; + case GenInlineItem::PChar: + ret << P(); + break; + case GenInlineItem::Char: + ret << GET_KEY(); + break; + case GenInlineItem::Hold: + ret << P() << "--;"; + break; + case GenInlineItem::Exec: + EXEC( ret, item, targState, inFinish ); + break; + case GenInlineItem::Curs: + CURS( ret, inFinish ); + break; + case GenInlineItem::Targs: + TARGS( ret, inFinish, targState ); + break; + case GenInlineItem::Entry: + ret << item->targState->id; + break; + case GenInlineItem::GotoExpr: + GOTO_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::CallExpr: + CALL_EXPR( ret, item, targState, inFinish ); + break; + case GenInlineItem::NextExpr: + NEXT_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::LmSwitch: + LM_SWITCH( ret, item, targState, inFinish ); + break; + case GenInlineItem::LmSetActId: + SET_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokEnd: + SET_TOKEND( ret, item ); + break; + case GenInlineItem::LmGetTokEnd: + GET_TOKEND( ret, item ); + break; + case GenInlineItem::LmInitTokStart: + INIT_TOKSTART( ret, item ); + break; + case GenInlineItem::LmInitAct: + INIT_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokStart: + SET_TOKSTART( ret, item ); + break; + case GenInlineItem::SubAction: + SUB_ACTION( ret, item, targState, inFinish ); + break; + case GenInlineItem::Break: + BREAK( ret, targState ); + break; + } + } +} +/* Write out paths in line directives. Escapes any special characters. */ +string CSharpFsmCodeGen::LDIR_PATH( char *path ) +{ + ostringstream ret; + for ( char *pc = path; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + ret << "\\\\"; + else + ret << *pc; + } + return ret.str(); +} + +void CSharpFsmCodeGen::ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ) +{ + /* Write the preprocessor line info for going into the source file. */ + csharpLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + ret << "\t{"; + INLINE_LIST( ret, action->inlineList, targState, inFinish ); + ret << "}\n"; +} + +void CSharpFsmCodeGen::CONDITION( ostream &ret, GenAction *condition ) +{ + ret << "\n"; + csharpLineDirective( ret, condition->loc.fileName, condition->loc.line ); + INLINE_LIST( ret, condition->inlineList, 0, false ); +} + +string CSharpFsmCodeGen::ERROR_STATE() +{ + ostringstream ret; + if ( redFsm->errState != 0 ) + ret << redFsm->errState->id; + else + ret << "-1"; + return ret.str(); +} + +string CSharpFsmCodeGen::FIRST_FINAL_STATE() +{ + ostringstream ret; + if ( redFsm->firstFinState != 0 ) + ret << redFsm->firstFinState->id; + else + ret << redFsm->nextStateId; + return ret.str(); +} + +void CSharpFsmCodeGen::writeInit() +{ + out << " {\n"; + + if ( !noCS ) + out << "\t" << vCS() << " = " << START() << ";\n"; + + /* If there are any calls, then the stack top needs initialization. */ + if ( redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\t" << TOP() << " = 0;\n"; + + if ( hasLongestMatch ) { + out << + " " << TOKSTART() << " = " << NULL_ITEM() << ";\n" + " " << TOKEND() << " = " << NULL_ITEM() << ";\n" + " " << ACT() << " = 0;\n"; + } + out << " }\n"; +} + +string CSharpFsmCodeGen::DATA_PREFIX() +{ + if ( !noPrefix ) + return FSM_NAME() + "_"; + return ""; +} + +/* Emit the alphabet data type. */ +string CSharpFsmCodeGen::ALPH_TYPE() +{ + string ret = keyOps->alphType->data1; + if ( keyOps->alphType->data2 != 0 ) { + ret += " "; + ret += + keyOps->alphType->data2; + } + return ret; +} + +/* Emit the alphabet data type. */ +string CSharpFsmCodeGen::WIDE_ALPH_TYPE() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + ret = ALPH_TYPE(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + assert( wideType != 0 ); + + ret = wideType->data1; + if ( wideType->data2 != 0 ) { + ret += " "; + ret += wideType->data2; + } + } + return ret; +} + +void CSharpFsmCodeGen::STATE_IDS() +{ + if ( redFsm->startState != 0 ) + STATIC_VAR( "int", START() ) << " = " << START_STATE_ID() << ";\n"; + + if ( !noFinal ) + STATIC_VAR( "int" , FIRST_FINAL() ) << " = " << FIRST_FINAL_STATE() << ";\n"; + + if ( !noError ) + STATIC_VAR( "int", ERROR() ) << " = " << ERROR_STATE() << ";\n"; + + out << "\n"; + + if ( !noEntry && entryPointNames.length() > 0 ) { + for ( EntryNameVect::Iter en = entryPointNames; en.lte(); en++ ) { + STATIC_VAR( "int", DATA_PREFIX() + "en_" + *en ) << + " = " << entryPointIds[en.pos()] << ";\n"; + } + out << "\n"; + } +} + + +void CSharpFsmCodeGen::writeStart() +{ + out << START_STATE_ID(); +} + +void CSharpFsmCodeGen::writeFirstFinal() +{ + out << FIRST_FINAL_STATE(); +} + +void CSharpFsmCodeGen::writeError() +{ + out << ERROR_STATE(); +} + +/* + * C# Specific + */ +string CSharpCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + if ( dataExpr == 0 ) + ret << "data"; + else + INLINE_LIST( ret, dataExpr, 0, false ); + + ret << "[" << P() << "]"; + } + return ret.str(); +} +string CSharpCodeGen::NULL_ITEM() +{ + return "-1"; +} + +string CSharpCodeGen::POINTER() +{ + // XXX C# has no pointers + // multiple items seperated by commas can also be pointer types. + return " "; +} + +string CSharpCodeGen::PTR_CONST() +{ + return ""; +} + +std::ostream &CSharpCodeGen::OPEN_ARRAY( string type, string name ) +{ + out << "static readonly " << type << "[] " << name << " = "; + /* + if (type == "char") + out << "Encoding.ASCII.Get"; + else */ + out << "new " << type << " [] {\n"; + return out; +} + +std::ostream &CSharpCodeGen::CLOSE_ARRAY() +{ + return out << "};\n"; +} + +std::ostream &CSharpCodeGen::STATIC_VAR( string type, string name ) +{ + out << "const " << type << " " << name; + return out; +} + +string CSharpCodeGen::ARR_OFF( string ptr, string offset ) +{ + // XXX C# can't do pointer arithmetic + return "&" + ptr + "[" + offset + "]"; +} + +string CSharpCodeGen::CAST( string type ) +{ + return "(" + type + ")"; +} + +string CSharpCodeGen::UINT( ) +{ + return "uint"; +} + +std::ostream &CSharpCodeGen::SWITCH_DEFAULT() +{ + out << " default: break;\n"; + return out; +} + +string CSharpCodeGen::CTRL_FLOW() +{ + return "if (true) "; +} + +void CSharpCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + out << "const " << ALPH_TYPE() << " " << DATA_PREFIX() << + "ex_" << ex->name << " = " << KEY(ex->key) << ";\n"; + } + out << "\n"; + } +} + +/* + * End C#-specific code. + */ + +void CSharpFsmCodeGen::finishRagelDef() +{ + if ( codeStyle == GenGoto || codeStyle == GenFGoto || + codeStyle == GenIpGoto || codeStyle == GenSplit ) + { + /* For directly executable machines there is no required state + * ordering. Choose a depth-first ordering to increase the + * potential for fall-throughs. */ + redFsm->depthFirstOrdering(); + } + else { + /* The frontend will do this for us, but it may be a good idea to + * force it if the intermediate file is edited. */ + redFsm->sortByStateId(); + } + + /* Choose default transitions and the single transition. */ + redFsm->chooseDefaultSpan(); + + /* Maybe do flat expand, otherwise choose single. */ + if ( codeStyle == GenFlat || codeStyle == GenFFlat ) + redFsm->makeFlat(); + else + redFsm->chooseSingle(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + if ( codeStyle == GenSplit ) + redFsm->partitionFsm( numSplitPartitions ); + + if ( codeStyle == GenIpGoto || codeStyle == GenSplit ) + redFsm->setInTrans(); + + /* Anlayze Machine will find the final action reference counts, among + * other things. We will use these in reporting the usage + * of fsm directives in action code. */ + analyzeMachine(); + + /* Determine if we should use indicies. */ + calcIndexSize(); +} + +ostream &CSharpFsmCodeGen::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &CSharpFsmCodeGen::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + diff --git a/contrib/tools/ragel6/cscodegen.h b/contrib/tools/ragel6/cscodegen.h new file mode 100644 index 0000000000..0471ea9164 --- /dev/null +++ b/contrib/tools/ragel6/cscodegen.h @@ -0,0 +1,205 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CSCODEGEN_H +#define _CSCODEGEN_H + +#include <iostream> +#include <string> +#include <stdio.h> +#include "common.h" +#include "gendata.h" + +using std::string; +using std::ostream; + +/* Integer array line length. */ +#define IALL 8 + +/* Forwards. */ +struct RedFsmAp; +struct RedStateAp; +struct CodeGenData; +struct GenAction; +struct NameInst; +struct GenInlineItem; +struct GenInlineList; +struct RedAction; +struct LongestMatch; +struct LongestMatchPart; + +string itoa( int i ); + +/* + * class CSharpFsmCodeGen + */ +class CSharpFsmCodeGen : public CodeGenData +{ +public: + CSharpFsmCodeGen( ostream &out ); + virtual ~CSharpFsmCodeGen() {} + + virtual void finishRagelDef(); + virtual void writeInit(); + virtual void writeStart(); + virtual void writeFirstFinal(); + virtual void writeError(); + +protected: + string FSM_NAME(); + string START_STATE_ID(); + ostream &ACTIONS_ARRAY(); + string GET_WIDE_KEY(); + string GET_WIDE_KEY( RedStateAp *state ); + string TABS( int level ); + string KEY( Key key ); + string ALPHA_KEY( Key key ); + string LDIR_PATH( char *path ); + void ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ); + void CONDITION( ostream &ret, GenAction *condition ); + string ALPH_TYPE(); + string WIDE_ALPH_TYPE(); + string ARRAY_TYPE( unsigned long maxVal ); + string ARRAY_TYPE( unsigned long maxVal, bool forceSigned ); + + virtual string ARR_OFF( string ptr, string offset ) = 0; + virtual string CAST( string type ) = 0; + virtual string UINT() = 0; + virtual string NULL_ITEM() = 0; + virtual string POINTER() = 0; + virtual string GET_KEY(); + virtual ostream &SWITCH_DEFAULT() = 0; + + string P(); + string PE(); + string vEOF(); + + string ACCESS(); + string vCS(); + string STACK(); + string TOP(); + string TOKSTART(); + string TOKEND(); + string ACT(); + + string DATA_PREFIX(); + string PM() { return "_" + DATA_PREFIX() + "partition_map"; } + string C() { return "_" + DATA_PREFIX() + "cond_spaces"; } + string CK() { return "_" + DATA_PREFIX() + "cond_keys"; } + string K() { return "_" + DATA_PREFIX() + "trans_keys"; } + string I() { return "_" + DATA_PREFIX() + "indicies"; } + string CO() { return "_" + DATA_PREFIX() + "cond_offsets"; } + string KO() { return "_" + DATA_PREFIX() + "key_offsets"; } + string IO() { return "_" + DATA_PREFIX() + "index_offsets"; } + string CL() { return "_" + DATA_PREFIX() + "cond_lengths"; } + string SL() { return "_" + DATA_PREFIX() + "single_lengths"; } + string RL() { return "_" + DATA_PREFIX() + "range_lengths"; } + string A() { return "_" + DATA_PREFIX() + "actions"; } + string TA() { return "_" + DATA_PREFIX() + "trans_actions"; } + string TT() { return "_" + DATA_PREFIX() + "trans_targs"; } + string TSA() { return "_" + DATA_PREFIX() + "to_state_actions"; } + string FSA() { return "_" + DATA_PREFIX() + "from_state_actions"; } + string EA() { return "_" + DATA_PREFIX() + "eof_actions"; } + string ET() { return "_" + DATA_PREFIX() + "eof_trans"; } + string SP() { return "_" + DATA_PREFIX() + "key_spans"; } + string CSP() { return "_" + DATA_PREFIX() + "cond_key_spans"; } + string START() { return DATA_PREFIX() + "start"; } + string ERROR() { return DATA_PREFIX() + "error"; } + string FIRST_FINAL() { return DATA_PREFIX() + "first_final"; } + string CTXDATA() { return DATA_PREFIX() + "ctxdata"; } + + void INLINE_LIST( ostream &ret, GenInlineList *inlineList, int targState, bool inFinish ); + virtual void GOTO( ostream &ret, int gotoDest, bool inFinish ) = 0; + virtual void CALL( ostream &ret, int callDest, int targState, bool inFinish ) = 0; + virtual void NEXT( ostream &ret, int nextDest, bool inFinish ) = 0; + virtual void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, + int targState, bool inFinish ) = 0; + virtual void RET( ostream &ret, bool inFinish ) = 0; + virtual void BREAK( ostream &ret, int targState ) = 0; + virtual void CURS( ostream &ret, bool inFinish ) = 0; + virtual void TARGS( ostream &ret, bool inFinish, int targState ) = 0; + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ); + void STATE_IDS(); + + string ERROR_STATE(); + string FIRST_FINAL_STATE(); + + virtual string PTR_CONST() = 0; + virtual ostream &OPEN_ARRAY( string type, string name ) = 0; + virtual ostream &CLOSE_ARRAY() = 0; + virtual ostream &STATIC_VAR( string type, string name ) = 0; + + virtual string CTRL_FLOW() = 0; + + ostream &source_warning(const InputLoc &loc); + ostream &source_error(const InputLoc &loc); + + unsigned int arrayTypeSize( unsigned long maxVal ); + + bool outLabelUsed; + bool testEofUsed; + bool againLabelUsed; + bool useIndicies; + +public: + /* Determine if we should use indicies. */ + virtual void calcIndexSize() {} + + void genLineDirective( ostream &out ); +}; + +class CSharpCodeGen : virtual public CSharpFsmCodeGen +{ +public: + CSharpCodeGen( ostream &out ) : CSharpFsmCodeGen(out) {} + + virtual string GET_KEY(); + virtual string NULL_ITEM(); + virtual string POINTER(); + virtual ostream &SWITCH_DEFAULT(); + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + virtual string ARR_OFF( string ptr, string offset ); + virtual string CAST( string type ); + virtual string UINT(); + virtual string PTR_CONST(); + virtual string CTRL_FLOW(); + + virtual void writeExports(); +}; + +#define MAX(a, b) (a > b ? a : b) + +#endif diff --git a/contrib/tools/ragel6/csfflat.cpp b/contrib/tools/ragel6/csfflat.cpp new file mode 100644 index 0000000000..f504eb2606 --- /dev/null +++ b/contrib/tools/ragel6/csfflat.cpp @@ -0,0 +1,389 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csfflat.h" +#include "redfsm.h" +#include "gendata.h" + +std::ostream &CSharpFFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &CSharpFFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &CSharpFFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + +/* Write out the function for a transition. */ +std::ostream &CSharpFFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpFFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFFlatCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void CSharpFFlatCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpFFlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " {\n" + " " << slenType << " _slen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << ";\n"; + out << " " << transType << " _trans"; + + if ( redFsm->anyConditions() ) + out << ", _cond"; + + out << ";\n"; + + out << + " " << "int _keys;\n" + " " << indsType << " _inds;\n"; + /* + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxIndex) << POINTER() << "_inds;\n";*/ + + if ( redFsm->anyConditions() ) { + out << + " " << condsType << " _conds;\n" + " " << WIDE_ALPH_TYPE() << " _widec;\n"; + } + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " switch ( " << TA() << "[_trans] ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << CAST(transType) << " (" << ET() << + "[" << vCS() << "] - 1);\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/contrib/tools/ragel6/csfflat.h b/contrib/tools/ragel6/csfflat.h new file mode 100644 index 0000000000..b102fe5b7c --- /dev/null +++ b/contrib/tools/ragel6/csfflat.h @@ -0,0 +1,55 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CSFFLAT_H +#define _CSFFLAT_H + +#include <iostream> +#include "csflat.h" + +/* Forwards. */ +struct CodeGenData; + +/* + * CSharpFFlatCodeGen + */ +class CSharpFFlatCodeGen : public CSharpFlatCodeGen +{ +public: + CSharpFFlatCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpFlatCodeGen(out) {} +private: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/contrib/tools/ragel6/csfgoto.cpp b/contrib/tools/ragel6/csfgoto.cpp new file mode 100644 index 0000000000..5a26424c47 --- /dev/null +++ b/contrib/tools/ragel6/csfgoto.cpp @@ -0,0 +1,296 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csfgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +std::ostream &CSharpFGotoCodeGen::EXEC_ACTIONS() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* We are at the start of a glob, write the case. */ + out << "f" << redAct->actListId << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tgoto _again;\n"; + } + } + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFGotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFGotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpFGotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &CSharpFGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\tcase " << st->id << ": "; + + /* Jump to the func. */ + out << "goto f" << st->eofAction->actListId << ";\n"; + } + } + + return out; +} + +unsigned int CSharpFGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + return act; +} + +unsigned int CSharpFGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + return act; +} + +unsigned int CSharpFGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + return act; +} + +void CSharpFGotoCodeGen::writeData() +{ + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpFGotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + out << + " switch ( " << vCS() << " ) {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_ACTIONS() << "\n"; + + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " switch ( " << vCS() << " ) {\n"; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + SWITCH_DEFAULT() << + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/contrib/tools/ragel6/csfgoto.h b/contrib/tools/ragel6/csfgoto.h new file mode 100644 index 0000000000..fa9447b685 --- /dev/null +++ b/contrib/tools/ragel6/csfgoto.h @@ -0,0 +1,55 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CSFGOTO_H +#define _CSFGOTO_H + +#include <iostream> +#include "csgoto.h" + +/* Forwards. */ +struct CodeGenData; + + +/* + * class CSharpFGotoCodeGen + */ +class CSharpFGotoCodeGen : public CSharpGotoCodeGen +{ +public: + CSharpFGotoCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpGotoCodeGen(out) {} + + std::ostream &EXEC_ACTIONS(); + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &FINISH_CASES(); + std::ostream &EOF_ACTION_SWITCH(); + unsigned int TO_STATE_ACTION( RedStateAp *state ); + unsigned int FROM_STATE_ACTION( RedStateAp *state ); + unsigned int EOF_ACTION( RedStateAp *state ); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/contrib/tools/ragel6/csflat.cpp b/contrib/tools/ragel6/csflat.cpp new file mode 100644 index 0000000000..4d5bafa124 --- /dev/null +++ b/contrib/tools/ragel6/csflat.cpp @@ -0,0 +1,890 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csflat.h" +#include "redfsm.h" +#include "gendata.h" + +std::ostream &CSharpFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &CSharpFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &CSharpFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + +std::ostream &CSharpFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &CSharpFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &CSharpFlatCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &CSharpFlatCodeGen::FLAT_INDEX_OFFSET() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + if ( st->transList != 0 ) + curIndOffset += keyOps->span( st->lowKey, st->highKey ); + + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::KEY_SPANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->transList != 0 ) + span = keyOps->span( st->lowKey, st->highKey ); + out << span; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::TO_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::FROM_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::EOF_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::EOF_TRANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &CSharpFlatCodeGen::COND_KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just cond low key and cond high key. */ + out << ALPHA_KEY( st->condLowKey ) << ", "; + out << ALPHA_KEY( st->condHighKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + if ( keyOps->alphType->isChar ) + out << "(char) " << 0 << "\n"; + else + out << 0 << "\n"; + + return out; +} + +std::ostream &CSharpFlatCodeGen::COND_KEY_SPANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->condList != 0 ) + span = keyOps->span( st->condLowKey, st->condHighKey ); + out << span; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::CONDS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->condList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + if ( st->condList[pos] != 0 ) + out << st->condList[pos]->condSpaceId + 1 << ", "; + else + out << "0, "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::COND_INDEX_OFFSET() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + if ( st->condList != 0 ) + curIndOffset += keyOps->span( st->condLowKey, st->condHighKey ); + } + out << "\n"; + return out; +} + + +std::ostream &CSharpFlatCodeGen::KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just low key and high key. */ + out << ALPHA_KEY( st->lowKey ) << ", "; + out << ALPHA_KEY( st->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + if ( keyOps->alphType->isChar ) + out << "(char) " << 0 << "\n"; + else + out << 0 << "\n"; + + return out; +} + +std::ostream &CSharpFlatCodeGen::INDICIES() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->transList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + out << st->transList[pos]->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) + out << st->defTrans->id << ", "; + + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpFlatCodeGen::TRANS_TARGS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Record the position, need this for eofTrans. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id; + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalStates % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + + +std::ostream &CSharpFlatCodeGen::TRANS_ACTIONS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalAct % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + +void CSharpFlatCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << vCS() << "<<1;\n" + " _inds = " << IO() << "[" << vCS() << "];\n" + "\n" + " _slen = " << SP() << "[" << vCS() << "];\n" + " _trans = " << I() << "[_inds + (\n" + " _slen > 0 && " << K() << "[_keys] <=" << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= " << K() <<"[_keys+1] ?\n" + " " << GET_WIDE_KEY() << " - " << K() << "[_keys] : _slen ) ];\n" + "\n"; +} + +void CSharpFlatCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << "; " << + CTRL_FLOW() << "goto _again;}"; +} + +void CSharpFlatCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpFlatCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void CSharpFlatCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void CSharpFlatCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void CSharpFlatCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void CSharpFlatCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + + +void CSharpFlatCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + + +void CSharpFlatCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpFlatCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void CSharpFlatCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpFlatCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n"; + + out << + " _keys = " << vCS() << "<<1;\n" + " _conds = " << CO() << "[" << vCS() << "];\n" +// " _keys = " << ARR_OFF( CK(), "(" + vCS() + "<<1)" ) << ";\n" +// " _conds = " << ARR_OFF( C(), CO() + "[" + vCS() + "]" ) << ";\n" + "\n" + " _slen = " << CSP() << "[" << vCS() << "];\n" + " if (_slen > 0 && " << CK() << "[_keys] <=" + << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= " << CK() << "[_keys+1])\n" + " _cond = " << C() << "[_conds+" << GET_WIDE_KEY() << " - " << + CK() << "[_keys]];\n" + " else\n" + " _cond = 0;" + "\n"; + /* XXX This version of the code doesn't work because Mono is weird. Works + * fine in Microsoft's csc, even though the bug report filed claimed it + * didn't. + " _slen = " << CSP() << "[" << vCS() << "];\n" + " _cond = _slen > 0 && " << CK() << "[_keys] <=" + << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= " << CK() << "[_keys+1] ?\n" + " " << C() << "[_conds+" << GET_WIDE_KEY() << " - " << CK() + << "[_keys]] : 0;\n" + "\n"; + */ + out << + " switch ( _cond ) {\n"; + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId + 1 << ": {\n"; + out << TABS(2) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << " }\n"; + out << " break;\n"; + } + + SWITCH_DEFAULT(); + + out << + " }\n"; +} + +void CSharpFlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " {\n" + " " << slenType << " _slen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " " << transType << " _trans"; + + if ( redFsm->anyConditions() ) + out << ", _cond"; + out << ";\n"; + + if ( redFsm->anyToStateActions() || + redFsm->anyRegActions() || redFsm->anyFromStateActions() ) + { + out << + " int _acts;\n" + " int _nacts;\n"; + } + + out << + " " << "int _keys;\n" + " " << indsType << " _inds;\n"; + /* + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxIndex) << POINTER() << "_inds;\n";*/ + + if ( redFsm->anyConditions() ) { + out << + " " << condsType << " _conds;\n" + " " << WIDE_ALPH_TYPE() << " _widec;\n"; + } + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" << vCS() << "];\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " _acts = " << TA() << "[_trans];\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] )\n {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "];\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << CAST(transType) << " (" << ET() << + "[" << vCS() << "] - 1);\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxActArrItem) << + POINTER() << "__acts = " << + EA() << "[" << vCS() << "];\n" + " " << UINT() << " __nacts = " << CAST(UINT()) << " " << + A() << "[__acts++];\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( " << A() << "[__acts++] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} + +void CSharpFlatCodeGen::initVarTypes() +{ + slenType = ARRAY_TYPE(MAX(redFsm->maxSpan, redFsm->maxCondSpan)); + transType = ARRAY_TYPE(redFsm->maxIndex+1); + indsType = ARRAY_TYPE(redFsm->maxFlatIndexOffset); + condsType = ARRAY_TYPE(redFsm->maxCondIndexOffset); +} diff --git a/contrib/tools/ragel6/csflat.h b/contrib/tools/ragel6/csflat.h new file mode 100644 index 0000000000..a576cd953f --- /dev/null +++ b/contrib/tools/ragel6/csflat.h @@ -0,0 +1,91 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CSFLAT_H +#define _CSFLAT_H + +#include <iostream> +#include "cscodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; + +/* + * CSharpFlatCodeGen + */ +class CSharpFlatCodeGen : virtual public CSharpFsmCodeGen, public CSharpCodeGen +{ +public: + CSharpFlatCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpCodeGen(out) {} + virtual ~CSharpFlatCodeGen() { } + +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &FLAT_INDEX_OFFSET(); + std::ostream &KEY_SPANS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + void LOCATE_TRANS(); + + std::ostream &COND_INDEX_OFFSET(); + void COND_TRANSLATE(); + std::ostream &CONDS(); + std::ostream &COND_KEYS(); + std::ostream &COND_KEY_SPANS(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); + + void initVarTypes(); + string slenType, transType, indsType, condsType; +}; + +#endif diff --git a/contrib/tools/ragel6/csftable.cpp b/contrib/tools/ragel6/csftable.cpp new file mode 100644 index 0000000000..44378e8246 --- /dev/null +++ b/contrib/tools/ragel6/csftable.cpp @@ -0,0 +1,438 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csftable.h" +#include "redfsm.h" +#include "gendata.h" + +/* Determine if we should use indicies or not. */ +void CSharpFTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActListId) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActListId) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &CSharpFTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &CSharpFTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &CSharpFTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + + +/* Write out the function for a transition. */ +std::ostream &CSharpFTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpFTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &CSharpFTabCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\tcase " << redAct->actListId+1 << ":\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void CSharpFTabCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpFTabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " {\n" + " " << klenType << " _klen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " " << keysType << " _keys;\n" + " " << transType << " _trans;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch ( " << FSA() << "[" << vCS() << "] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + out << "_match:\n"; + + if ( useIndicies ) + out << " _trans = " << CAST(transType) << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " switch ( " << TA() << "[_trans] ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " switch ( " << TSA() << "[" << vCS() << "] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << CAST(transType) << " (" << ET() << + "[" << vCS() << "] - 1);\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch ( " << EA() << "[" << vCS() << "] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/contrib/tools/ragel6/csftable.h b/contrib/tools/ragel6/csftable.h new file mode 100644 index 0000000000..612ec32f7e --- /dev/null +++ b/contrib/tools/ragel6/csftable.h @@ -0,0 +1,56 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _CSFTABLE_H +#define _CSFTABLE_H + +#include <iostream> +#include "cstable.h" + +/* Forwards. */ +struct CodeGenData; + + +/* + * CSharpFTabCodeGen + */ +class CSharpFTabCodeGen : public CSharpTabCodeGen +{ +public: + CSharpFTabCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpTabCodeGen(out) {} +private: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void writeData(); + virtual void writeExec(); + virtual void calcIndexSize(); +}; + +#endif diff --git a/contrib/tools/ragel6/csgoto.cpp b/contrib/tools/ragel6/csgoto.cpp new file mode 100644 index 0000000000..4bb2ec1edf --- /dev/null +++ b/contrib/tools/ragel6/csgoto.cpp @@ -0,0 +1,801 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csgoto.h" +#include "redfsm.h" +#include "bstmap.h" +#include "gendata.h" + +/* Emit the goto to take for a given transition. */ +std::ostream &CSharpGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + out << TABS(level) << "goto tr" << trans->id << ";"; + return out; +} + +std::ostream &CSharpGotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpGotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpGotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpGotoCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +void CSharpGotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + /* Label the state. */ + out << "case " << state->id << ":\n"; +} + + +void CSharpGotoCodeGen::emitSingleSwitch( RedStateAp *state ) +{ + /* Load up the singles. */ + int numSingles = state->outSingle.length(); + RedTransEl *data = state->outSingle.data; + + if ( numSingles == 1 ) { + /* If there is a single single key then write it out as an if. */ + out << "\tif ( " << GET_WIDE_KEY(state) << " == " << + KEY(data[0].lowKey) << " )\n\t\t"; + + /* Virtual function for writing the target of the transition. */ + TRANS_GOTO(data[0].value, 0) << "\n"; + } + else if ( numSingles > 1 ) { + /* Write out single keys in a switch if there is more than one. */ + out << "\tswitch( " << GET_WIDE_KEY(state) << " ) {\n"; + + /* Write out the single indicies. */ + for ( int j = 0; j < numSingles; j++ ) { + out << "\t\tcase " << ALPHA_KEY(data[j].lowKey) << ": "; + TRANS_GOTO(data[j].value, 0) << "\n"; + } + + /* Emits a default case for D code. */ + SWITCH_DEFAULT(); + + /* Close off the transition switch. */ + out << "\t}\n"; + } +} + +void CSharpGotoCodeGen::emitRangeBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + RedTransEl *data = state->outRange.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid].lowKey == keyOps->minKey; + bool limitHigh = data[mid].highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " ) {\n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " ) {\n"; + emitRangeBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " ) {\n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " ) {\n"; + emitRangeBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "} else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_WIDE_KEY(state) << " >= " << + KEY(data[mid].lowKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " && " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } +} + +void CSharpGotoCodeGen::STATE_GOTO_ERROR() +{ + /* Label the state and bail immediately. */ + outLabelUsed = true; + RedStateAp *state = redFsm->errState; + out << "case " << state->id << ":\n"; + out << " goto _out;\n"; +} + +void CSharpGotoCodeGen::COND_TRANSLATE( GenStateCond *stateCond, int level ) +{ + GenCondSpace *condSpace = stateCond->condSpace; + out << TABS(level) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(level) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } +} + +void CSharpGotoCodeGen::emitCondBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + GenStateCond **data = state->stateCondVect.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid]->lowKey == keyOps->minKey; + bool limitHigh = data[mid]->highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if ( " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " ) {\n"; + emitCondBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "} else if ( " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " ) {\n"; + emitCondBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if ( " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " ) {\n"; + emitCondBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if ( " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " ) {\n"; + emitCondBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_KEY() << " >= " << + KEY(data[mid]->lowKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " && " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " )\n {"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + COND_TRANSLATE(data[mid], level); + } + } +} + +std::ostream &CSharpGotoCodeGen::STATE_GOTOS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* Writing code above state gotos. */ + GOTO_HEADER( st ); + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + } + } + return out; +} + +std::ostream &CSharpGotoCodeGen::TRANSITIONS() +{ + /* Emit any transitions that have functions and that go to + * this state. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Write the label for the transition so it can be jumped to. */ + out << " tr" << trans->id << ": "; + + /* Destination state. */ + if ( trans->action != 0 && trans->action->anyCurStateRef() ) + out << "_ps = " << vCS() << ";"; + out << vCS() << " = " << trans->targ->id << "; "; + + if ( trans->action != 0 ) { + /* Write out the transition func. */ + out << "goto f" << trans->action->actListId << ";\n"; + } + else { + /* No code to execute, just loop around. */ + out << "goto _again;\n"; + } + } + return out; +} + +std::ostream &CSharpGotoCodeGen::EXEC_FUNCS() +{ + /* Make labels that set acts and jump to execFuncs. Loop func indicies. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + out << " f" << redAct->actListId << ": " << + "_acts = " << itoa( redAct->location+1 ) << ";" + " goto execFuncs;\n"; + } + } + + out << + "\n" + "execFuncs:\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + " goto _again;\n"; + return out; +} + +unsigned int CSharpGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +unsigned int CSharpGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +unsigned int CSharpGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + +std::ostream &CSharpGotoCodeGen::TO_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = TO_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &CSharpGotoCodeGen::FROM_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = FROM_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &CSharpGotoCodeGen::EOF_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = EOF_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &CSharpGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\tcase " << st->id << ": "; + + /* Write the goto func. */ + out << "goto f" << st->eofAction->actListId << ";\n"; + } + } + + return out; +} + +void CSharpGotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << "; " << + CTRL_FLOW() << "goto _again;}"; +} + +void CSharpGotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpGotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void CSharpGotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void CSharpGotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void CSharpGotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void CSharpGotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpGotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpGotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpGotoCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void CSharpGotoCodeGen::writeData() +{ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpGotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " " << ARRAY_TYPE(redFsm->maxActionLoc) << " _acts;\n" + " " << ARRAY_TYPE(redFsm->maxActArrItem) << " _nacts;\n"; + } + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + out << "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" << vCS() << "];\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + out << + " switch ( " << vCS() << " ) {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_FUNCS() << "\n"; + + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "];\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " switch ( " << vCS() << " ) {\n"; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + SWITCH_DEFAULT() << + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " " << ARRAY_TYPE(redFsm->maxActionLoc) << " __acts = " << + EA() << "[" << vCS() << "];\n" + " " << ARRAY_TYPE(redFsm->maxActArrItem) << " __nacts = " << + A() << "[__acts++];\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( " << A() << "[__acts++] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} diff --git a/contrib/tools/ragel6/csgoto.h b/contrib/tools/ragel6/csgoto.h new file mode 100644 index 0000000000..749188408d --- /dev/null +++ b/contrib/tools/ragel6/csgoto.h @@ -0,0 +1,89 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOTOCODEGEN_H +#define _GOTOCODEGEN_H + +#include <iostream> +#include "cscodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; +struct GenStateCond; + +/* + * Goto driven fsm. + */ +class CSharpGotoCodeGen : virtual public CSharpFsmCodeGen, public CSharpCodeGen +{ +public: + CSharpGotoCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpCodeGen(out) {} + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &STATE_GOTOS(); + std::ostream &TRANSITIONS(); + std::ostream &EXEC_FUNCS(); + std::ostream &FINISH_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + virtual unsigned int TO_STATE_ACTION( RedStateAp *state ); + virtual unsigned int FROM_STATE_ACTION( RedStateAp *state ); + virtual unsigned int EOF_ACTION( RedStateAp *state ); + + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + + void COND_TRANSLATE( GenStateCond *stateCond, int level ); + void emitCondBSearch( RedStateAp *state, int level, int low, int high ); + void STATE_CONDS( RedStateAp *state, bool genDefault ); + + virtual std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + + void emitSingleSwitch( RedStateAp *state ); + void emitRangeBSearch( RedStateAp *state, int level, int low, int high ); + + /* Called from STATE_GOTOS just before writing the gotos */ + virtual void GOTO_HEADER( RedStateAp *state ); + virtual void STATE_GOTO_ERROR(); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/contrib/tools/ragel6/csipgoto.cpp b/contrib/tools/ragel6/csipgoto.cpp new file mode 100644 index 0000000000..0e3168a200 --- /dev/null +++ b/contrib/tools/ragel6/csipgoto.cpp @@ -0,0 +1,438 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "csipgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +bool CSharpIpGotoCodeGen::useAgainLabel() +{ + return redFsm->anyRegActionRets() || + redFsm->anyRegActionByValControl() || + redFsm->anyRegNextStmt(); +} + +void CSharpIpGotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << CTRL_FLOW() << "goto st" << gotoDest << ";}"; +} + +void CSharpIpGotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << targState << + "; " << CTRL_FLOW() << "goto st" << callDest << ";}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpIpGotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << targState << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpIpGotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpIpGotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "goto _again;}"; +} + +void CSharpIpGotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void CSharpIpGotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void CSharpIpGotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void CSharpIpGotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << targState; +} + +void CSharpIpGotoCodeGen::BREAK( ostream &ret, int targState ) +{ + /* FIXME: If this code generator is made active then BREAK generation + * needs to check csForced. */ + outLabelUsed = true; + ret << "{" << P() << "++; " << vCS() << " = " << targState << + "; " << CTRL_FLOW() << "goto _out;}"; +} + +bool CSharpIpGotoCodeGen::IN_TRANS_ACTIONS( RedStateAp *state ) +{ + bool anyWritten = false; + + /* Emit any transitions that have actions and that go to this state. */ + for ( int it = 0; it < state->numInTrans; it++ ) { + RedTransAp *trans = state->inTrans[it]; + if ( trans->action != 0 && trans->labelNeeded ) { + /* Remember that we wrote an action so we know to write the + * line directive for going back to the output. */ + anyWritten = true; + + /* Write the label for the transition so it can be jumped to. */ + out << "tr" << trans->id << ":\n"; + + /* If the action contains a next, then we must preload the current + * state since the action may or may not set it. */ + if ( trans->action->anyNextStmt() ) + out << " " << vCS() << " = " << trans->targ->id << ";\n"; + + /* Write each action in the list. */ + for ( GenActionTable::Iter item = trans->action->key; item.lte(); item++ ) + ACTION( out, item->value, trans->targ->id, false ); + + /* If the action contains a next then we need to reload, otherwise + * jump directly to the target state. */ + if ( trans->action->anyNextStmt() ) + out << "\tgoto _again;\n"; + else + out << "\tgoto st" << trans->targ->id << ";\n"; + } + } + + return anyWritten; +} + +/* Called from GotoCodeGen::STATE_GOTOS just before writing the gotos for each + * state. */ +void CSharpIpGotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + bool anyWritten = IN_TRANS_ACTIONS( state ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + if ( state->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->toStateAction->key; item.lte(); item++ ) + ACTION( out, item->value, state->id, false ); + } + + /* Advance and test buffer pos. */ + if ( state->labelNeeded ) { + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _test_eof" << state->id << ";\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + } + + /* Give the state a switch case. */ + out << "case " << state->id << ":\n"; + + if ( state->fromStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->fromStateAction->key; item.lte(); item++ ) + ACTION( out, item->value, state->id, false ); + } + + if ( anyWritten ) + genLineDirective( out ); + + /* Record the prev state if necessary. */ + if ( state->anyRegCurStateRef() ) + out << " _ps = " << state->id << ";\n"; +} + +void CSharpIpGotoCodeGen::STATE_GOTO_ERROR() +{ + /* In the error state we need to emit some stuff that usually goes into + * the header. */ + RedStateAp *state = redFsm->errState; + bool anyWritten = IN_TRANS_ACTIONS( state ); + + /* No case label needed since we don't switch on the error state. */ + if ( anyWritten ) + genLineDirective( out ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + /* Break out here. */ + outLabelUsed = true; + out << vCS() << " = " << state->id << ";\n"; + out << " goto _out;\n"; +} + + +/* Emit the goto to take for a given transition. */ +std::ostream &CSharpIpGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto tr" << trans->id << ";"; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto st" << trans->targ->id << ";"; + } + return out; +} + +std::ostream &CSharpIpGotoCodeGen::EXIT_STATES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->outNeeded ) { + testEofUsed = true; + out << " _test_eof" << st->id << ": " << vCS() << " = " << + st->id << "; goto _test_eof; \n"; + } + } + return out; +} + +std::ostream &CSharpIpGotoCodeGen::AGAIN_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + out << + " case " << st->id << ": goto st" << st->id << ";\n"; + } + return out; +} + +std::ostream &CSharpIpGotoCodeGen::FINISH_CASES() +{ + bool anyWritten = false; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofAction != 0 ) { + if ( st->eofAction->eofRefs == 0 ) + st->eofAction->eofRefs = new IntSet; + st->eofAction->eofRefs->insert( st->id ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " case " << st->id << ": goto tr" << st->eofTrans->id << ";\n"; + } + + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + if ( act->eofRefs != 0 ) { + for ( IntSet::Iter pst = *act->eofRefs; pst.lte(); pst++ ) + out << " case " << *pst << ": \n"; + + /* Remember that we wrote a trans so we know to write the + * line directive for going back to the output. */ + anyWritten = true; + + /* Write each action in the eof action list. */ + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) + ACTION( out, item->value, STATE_ERR_STATE, true ); + out << "\tbreak;\n"; + } + } + + if ( anyWritten ) + genLineDirective( out ); + return out; +} + +void CSharpIpGotoCodeGen::setLabelsNeeded( GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: { + /* Mark the target as needing a label. */ + item->targState->labelNeeded = true; + break; + } + default: break; + } + + if ( item->children != 0 ) + setLabelsNeeded( item->children ); + } +} + +/* Set up labelNeeded flag for each state. */ +void CSharpIpGotoCodeGen::setLabelsNeeded() +{ + /* If we use the _again label, then we the _again switch, which uses all + * labels. */ + if ( useAgainLabel() ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = true; + } + else { + /* Do not use all labels by default, init all labelNeeded vars to false. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = false; + + /* Walk all transitions and set only those that have targs. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* If there is no action with a next statement, then the label will be + * needed. */ + if ( trans->action == 0 || !trans->action->anyNextStmt() ) + trans->targ->labelNeeded = true; + + /* Need labels for states that have goto or calls in action code + * invoked on characters (ie, not from out action code). */ + if ( trans->action != 0 ) { + /* Loop the actions. */ + for ( GenActionTable::Iter act = trans->action->key; act.lte(); act++ ) { + /* Get the action and walk it's tree. */ + setLabelsNeeded( act->value->inlineList ); + } + } + } + } + + if ( !noEnd ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st != redFsm->errState ) + st->outNeeded = st->labelNeeded; + } + } +} + +void CSharpIpGotoCodeGen::writeData() +{ + STATE_IDS(); +} + +void CSharpIpGotoCodeGen::writeExec() +{ + /* Must set labels immediately before writing because we may depend on the + * noend write option. */ + setLabelsNeeded(); + testEofUsed = false; + outLabelUsed = false; + + out << " {\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( useAgainLabel() ) { + out << + " goto _resume;\n" + "\n" + "_again:\n" + " switch ( " << vCS() << " ) {\n"; + AGAIN_CASES() << + " default: break;\n" + " }\n" + "\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << "_resume:\n"; + } + + out << + " switch ( " << vCS() << " )\n {\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " }\n"; + EXIT_STATES() << + "\n"; + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n" + " switch ( " << vCS() << " ) {\n"; + FINISH_CASES(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << + " }\n"; +} diff --git a/contrib/tools/ragel6/csipgoto.h b/contrib/tools/ragel6/csipgoto.h new file mode 100644 index 0000000000..4aeb47d97d --- /dev/null +++ b/contrib/tools/ragel6/csipgoto.h @@ -0,0 +1,75 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _IPGCODEGEN_H +#define _IPGCODEGEN_H + +#include <iostream> +#include "csgoto.h" + +/* Forwards. */ +struct CodeGenData; + +/* + * class CSharpIpGotoCodeGen + */ +class CSharpIpGotoCodeGen : public CSharpGotoCodeGen +{ +public: + CSharpIpGotoCodeGen( ostream &out ) : CSharpFsmCodeGen(out), + CSharpGotoCodeGen(out) {} + + std::ostream &EXIT_STATES(); + std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + std::ostream &FINISH_CASES(); + std::ostream &AGAIN_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &ret, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void BREAK( ostream &ret, int targState ); + + virtual void writeData(); + virtual void writeExec(); + +protected: + bool useAgainLabel(); + + /* Called from GotoCodeGen::STATE_GOTOS just before writing the gotos for + * each state. */ + bool IN_TRANS_ACTIONS( RedStateAp *state ); + void GOTO_HEADER( RedStateAp *state ); + void STATE_GOTO_ERROR(); + + /* Set up labelNeeded flag for each state. */ + void setLabelsNeeded( GenInlineList *inlineList ); + void setLabelsNeeded(); +}; + +#endif diff --git a/contrib/tools/ragel6/cssplit.cpp b/contrib/tools/ragel6/cssplit.cpp new file mode 100644 index 0000000000..8c7464f09e --- /dev/null +++ b/contrib/tools/ragel6/cssplit.cpp @@ -0,0 +1,518 @@ +/* + * Copyright 2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + + +#include "ragel.h" +#include "cssplit.h" +#include "gendata.h" +#include <assert.h> + +using std::ostream; +using std::ios; +using std::endl; + +/* Emit the goto to take for a given transition. */ +std::ostream &CSharpSplitCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + if ( trans->targ->partition == currentPartition ) { + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto tr" << trans->id << ";"; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto st" << trans->targ->id << ";"; + } + } + else { + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto ptr" << trans->id << ";"; + trans->partitionBoundary = true; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto pst" << trans->targ->id << ";"; + trans->targ->partitionBoundary = true; + } + } + return out; +} + +/* Called from before writing the gotos for each state. */ +void CSharpSplitCodeGen::GOTO_HEADER( RedStateAp *state, bool stateInPartition ) +{ + bool anyWritten = IN_TRANS_ACTIONS( state ); + + if ( state->labelNeeded ) + out << "st" << state->id << ":\n"; + + if ( state->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->toStateAction->key; item.lte(); item++ ) + ACTION( out, item->value, state->id, false ); + } + + /* Advance and test buffer pos. */ + if ( state->labelNeeded ) { + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out" << state->id << ";\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + } + + /* Give the state a switch case. */ + out << "case " << state->id << ":\n"; + + if ( state->fromStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->fromStateAction->key; item.lte(); item++ ) + ACTION( out, item->value, state->id, false ); + } + + if ( anyWritten ) + genLineDirective( out ); + + /* Record the prev state if necessary. */ + if ( state->anyRegCurStateRef() ) + out << " _ps = " << state->id << ";\n"; +} + +std::ostream &CSharpSplitCodeGen::STATE_GOTOS( int partition ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partition == partition ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* We call into the base of the goto which calls back into us + * using virtual functions. Set the current partition rather + * than coding parameter passing throughout. */ + currentPartition = partition; + + /* Writing code above state gotos. */ + GOTO_HEADER( st, st->partition == partition ); + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + } + } + } + return out; +} + + +std::ostream &CSharpSplitCodeGen::PART_TRANS( int partition ) +{ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + if ( trans->partitionBoundary ) { + out << + "ptr" << trans->id << ":\n"; + + if ( trans->action != 0 ) { + /* If the action contains a next, then we must preload the current + * state since the action may or may not set it. */ + if ( trans->action->anyNextStmt() ) + out << " " << vCS() << " = " << trans->targ->id << ";\n"; + + /* Write each action in the list. */ + for ( GenActionTable::Iter item = trans->action->key; item.lte(); item++ ) + ACTION( out, item->value, trans->targ->id, false ); + } + + out << + " goto pst" << trans->targ->id << ";\n"; + trans->targ->partitionBoundary = true; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partitionBoundary ) { + out << + " pst" << st->id << ":\n" + " " << vCS() << " = " << st->id << ";\n"; + + if ( st->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + for ( GenActionTable::Iter item = st->toStateAction->key; item.lte(); item++ ) + ACTION( out, item->value, st->id, false ); + genLineDirective( out ); + } + + ptOutLabelUsed = true; + out << " goto _pt_out; \n"; + } + } + return out; +} + +std::ostream &CSharpSplitCodeGen::EXIT_STATES( int partition ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->partition == partition && st->outNeeded ) { + outLabelUsed = true; + out << " _out" << st->id << ": " << vCS() << " = " << + st->id << "; goto _out; \n"; + } + } + return out; +} + + +std::ostream &CSharpSplitCodeGen::PARTITION( int partition ) +{ + outLabelUsed = false; + ptOutLabelUsed = false; + + /* Initialize the partition boundaries, which get set during the writing + * of states. After the state writing we will */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + trans->partitionBoundary = false; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->partitionBoundary = false; + + out << " " << ALPH_TYPE() << " *p = *_pp, *pe = *_ppe;\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " int _ps = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( useAgainLabel() ) { + out << + " goto _resume;\n" + "\n" + "_again:\n" + " switch ( " << vCS() << " ) {\n"; + AGAIN_CASES() << + " default: break;\n" + " }\n" + "\n"; + + + if ( !noEnd ) { + outLabelUsed = true; + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out;\n"; + + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << + "_resume:\n"; + } + + out << + " switch ( " << vCS() << " )\n {\n"; + STATE_GOTOS( partition ); + SWITCH_DEFAULT() << + " }\n"; + PART_TRANS( partition ); + EXIT_STATES( partition ); + + if ( outLabelUsed ) { + out << + "\n" + " _out:\n" + " *_pp = p;\n" + " *_ppe = pe;\n" + " return 0;\n"; + } + + if ( ptOutLabelUsed ) { + out << + "\n" + " _pt_out:\n" + " *_pp = p;\n" + " *_ppe = pe;\n" + " return 1;\n"; + } + + return out; +} + +std::ostream &CSharpSplitCodeGen::PART_MAP() +{ + int *partMap = new int[redFsm->stateList.length()]; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + partMap[st->id] = st->partition; + + out << "\t"; + int totalItem = 0; + for ( int i = 0; i < redFsm->stateList.length(); i++ ) { + out << partMap[i]; + if ( i != redFsm->stateList.length() - 1 ) { + out << ", "; + if ( ++totalItem % IALL == 0 ) + out << "\n\t"; + } + } + + delete[] partMap; + return out; +} + +void CSharpSplitCodeGen::writeData() +{ + out << + "const int " << START() << " = " << START_STATE_ID() << ";\n" + "\n"; + + if ( !noFinal ) { + out << + "const int " << FIRST_FINAL() << " = " << FIRST_FINAL_STATE() << ";\n" + "\n"; + } + + if ( !noError ) { + out << + "const int " << ERROR() << " = " << ERROR_STATE() << ";\n" + "\n"; + } + + + OPEN_ARRAY( ARRAY_TYPE(numSplitPartitions), PM() ); + PART_MAP(); + CLOSE_ARRAY() << + "\n"; + + for ( int p = 0; p < redFsm->nParts; p++ ) { + out << "int partition" << p << "( " << ALPH_TYPE() << " **_pp, " << ALPH_TYPE() << + " **_ppe, struct " << FSM_NAME() << " *fsm );\n"; + } + out << "\n"; +} + +std::ostream &CSharpSplitCodeGen::ALL_PARTITIONS() +{ + /* compute the format string. */ + int width = 0, high = redFsm->nParts - 1; + while ( high > 0 ) { + width++; + high /= 10; + } + assert( width <= 8 ); + char suffFormat[] = "_%6.6d.c"; + suffFormat[2] = suffFormat[4] = ( '0' + width ); + + for ( int p = 0; p < redFsm->nParts; p++ ) { + char suffix[10]; + sprintf( suffix, suffFormat, p ); + const char *fn = fileNameFromStem( sourceFileName, suffix ); + const char *include = fileNameFromStem( sourceFileName, ".h" ); + + /* Create the filter on the output and open it. */ + output_filter *partFilter = new output_filter( fn ); + partFilter->open( fn, ios::out|ios::trunc ); + if ( !partFilter->is_open() ) { + error() << "error opening " << fn << " for writing" << endl; + exit(1); + } + + /* Attach the new file to the output stream. */ + std::streambuf *prev_rdbuf = out.rdbuf( partFilter ); + + out << + "#include \"" << include << "\"\n" + "int partition" << p << "( " << ALPH_TYPE() << " **_pp, " << ALPH_TYPE() << + " **_ppe, struct " << FSM_NAME() << " *fsm )\n" + "{\n"; + PARTITION( p ) << + "}\n\n"; + out.flush(); + + /* Fix the output stream. */ + out.rdbuf( prev_rdbuf ); + } + return out; +} + + +void CSharpSplitCodeGen::writeExec() +{ + /* Must set labels immediately before writing because we may depend on the + * noend write option. */ + setLabelsNeeded(); + out << + " {\n" + " int _stat = 0;\n"; + + if ( !noEnd ) { + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _out;\n"; + } + + out << " goto _resume;\n"; + + /* In this reentry, to-state actions have already been executed on the + * partition-switch exit from the last partition. */ + out << "_reenter:\n"; + + if ( !noEnd ) { + out << + " if ( ++" << P() << " == " << PE() << " )\n" + " goto _out;\n"; + } + else { + out << + " " << P() << " += 1;\n"; + } + + out << "_resume:\n"; + + out << + " switch ( " << PM() << "[" << vCS() << "] ) {\n"; + for ( int p = 0; p < redFsm->nParts; p++ ) { + out << + " case " << p << ":\n" + " _stat = partition" << p << "( &p, &pe, fsm );\n" + " break;\n"; + } + out << + " }\n" + " if ( _stat )\n" + " goto _reenter;\n"; + + if ( !noEnd ) + out << " _out: {}\n"; + + out << + " }\n"; + + ALL_PARTITIONS(); +} + +void CSharpSplitCodeGen::setLabelsNeeded( RedStateAp *fromState, GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: { + /* In split code gen we only need labels for transitions across + * partitions. */ + if ( fromState->partition == item->targState->partition ){ + /* Mark the target as needing a label. */ + item->targState->labelNeeded = true; + } + break; + } + default: break; + } + + if ( item->children != 0 ) + setLabelsNeeded( fromState, item->children ); + } +} + +void CSharpSplitCodeGen::setLabelsNeeded( RedStateAp *fromState, RedTransAp *trans ) +{ + /* In the split code gen we don't need labels for transitions across + * partitions. */ + if ( fromState->partition == trans->targ->partition ) { + /* If there is no action with a next statement, then the label will be + * needed. */ + trans->labelNeeded = true; + if ( trans->action == 0 || !trans->action->anyNextStmt() ) + trans->targ->labelNeeded = true; + } + + /* Need labels for states that have goto or calls in action code + * invoked on characters (ie, not from out action code). */ + if ( trans->action != 0 ) { + /* Loop the actions. */ + for ( GenActionTable::Iter act = trans->action->key; act.lte(); act++ ) { + /* Get the action and walk it's tree. */ + setLabelsNeeded( fromState, act->value->inlineList ); + } + } +} + +/* Set up labelNeeded flag for each state. */ +void CSharpSplitCodeGen::setLabelsNeeded() +{ + /* If we use the _again label, then we the _again switch, which uses all + * labels. */ + if ( useAgainLabel() ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = true; + } + else { + /* Do not use all labels by default, init all labelNeeded vars to false. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = false; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + trans->labelNeeded = false; + + /* Walk all transitions and set only those that have targs. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + for ( RedTransList::Iter tel = st->outRange; tel.lte(); tel++ ) + setLabelsNeeded( st, tel->value ); + + for ( RedTransList::Iter tel = st->outSingle; tel.lte(); tel++ ) + setLabelsNeeded( st, tel->value ); + + if ( st->defTrans != 0 ) + setLabelsNeeded( st, st->defTrans ); + } + } + + if ( !noEnd ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->outNeeded = st->labelNeeded; + } + else { + if ( redFsm->errState != 0 ) + redFsm->errState->outNeeded = true; + + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Any state with a transition in that has a break will need an + * out label. */ + if ( trans->action != 0 && trans->action->anyBreakStmt() ) + trans->targ->outNeeded = true; + } + } +} + diff --git a/contrib/tools/ragel6/cssplit.h b/contrib/tools/ragel6/cssplit.h new file mode 100644 index 0000000000..9ff2d8f7eb --- /dev/null +++ b/contrib/tools/ragel6/cssplit.h @@ -0,0 +1,53 @@ +/* + * Copyright 2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _SPLITCODEGEN_H +#define _SPLITCODEGEN_H + +#include "csipgoto.h" + +class CSharpSplitCodeGen : public CSharpIpGotoCodeGen +{ +public: + CSharpSplitCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpIpGotoCodeGen(out) {} + + bool ptOutLabelUsed; + + std::ostream &PART_MAP(); + std::ostream &EXIT_STATES( int partition ); + std::ostream &PART_TRANS( int partition ); + std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + void GOTO_HEADER( RedStateAp *state, bool stateInPartition ); + std::ostream &STATE_GOTOS( int partition ); + std::ostream &PARTITION( int partition ); + std::ostream &ALL_PARTITIONS(); + void writeData(); + void writeExec(); + void writeParts(); + + void setLabelsNeeded( RedStateAp *fromState, GenInlineList *inlineList ); + void setLabelsNeeded( RedStateAp *fromState, RedTransAp *trans ); + void setLabelsNeeded(); + + int currentPartition; +}; + +#endif diff --git a/contrib/tools/ragel6/cstable.cpp b/contrib/tools/ragel6/cstable.cpp new file mode 100644 index 0000000000..13c9e738ca --- /dev/null +++ b/contrib/tools/ragel6/cstable.cpp @@ -0,0 +1,1144 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "cstable.h" +#include "redfsm.h" +#include "gendata.h" + +/* Determine if we should use indicies or not. */ +void CSharpTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActionLoc) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActionLoc) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &CSharpTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &CSharpTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &CSharpTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + + +std::ostream &CSharpTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &CSharpTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &CSharpTabCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &CSharpTabCodeGen::COND_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->stateCondList.length(); + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::KEY_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->outSingle.length() + st->outRange.length()*2; + } + out << "\n"; + return out; +} + + +std::ostream &CSharpTabCodeGen::INDEX_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + curIndOffset += st->outSingle.length() + st->outRange.length(); + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::COND_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->stateCondList.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &CSharpTabCodeGen::SINGLE_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->outSingle.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::RANGE_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit length of range index. */ + out << st->outRange.length(); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::TO_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::FROM_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::EOF_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::EOF_TRANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + if ( !st.last() ) { + out << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &CSharpTabCodeGen::COND_KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Lower key. */ + out << ALPHA_KEY( sc->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + /* Upper key. */ + out << ALPHA_KEY( sc->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + if ( keyOps->alphType->isChar ) + out << "(char) " << 0 << "\n"; + else + out << 0 << "\n"; + + return out; +} + +std::ostream &CSharpTabCodeGen::COND_SPACES() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Cond Space id. */ + out << sc->condSpace->condSpaceId << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << ALPHA_KEY( stel->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Loop the state's transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + /* Lower key. */ + out << ALPHA_KEY( rtel->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + /* Upper key. */ + out << ALPHA_KEY( rtel->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + if ( keyOps->alphType->isChar ) + out << "(char) " << 0 << "\n"; + else + out << 0 << "\n"; + + return out; +} + +std::ostream &CSharpTabCodeGen::INDICIES() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << stel->value->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + out << rtel->value->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + out << st->defTrans->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::TRANS_TARGS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default target state. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + trans->pos = totalTrans; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + + +std::ostream &CSharpTabCodeGen::TRANS_ACTIONS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &CSharpTabCodeGen::TRANS_TARGS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Record the position, need this for eofTrans. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id; + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalStates % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + + +std::ostream &CSharpTabCodeGen::TRANS_ACTIONS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + if ( t < redFsm->transSet.length()-1 ) { + out << ", "; + if ( ++totalAct % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + +void CSharpTabCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << ";"; + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void CSharpTabCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void CSharpTabCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void CSharpTabCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void CSharpTabCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void CSharpTabCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void CSharpTabCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{"; + + ret << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << callDest << ";"; + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpTabCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{"; + + ret << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << ");"; + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void CSharpTabCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{"; + + ret << vCS() << " = " << STACK() << "[--" << TOP() << "]; "; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + ret << CTRL_FLOW() << "goto _again;"; + + ret << "}"; +} + +void CSharpTabCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; " << CTRL_FLOW() << "goto _out; }"; +} + +void CSharpTabCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void CSharpTabCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << KO() + "[" + vCS() + "]" << ";\n" + " _trans = " << CAST(transType) << IO() << "[" << vCS() << "];\n" + "\n" + " _klen = " << SL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " " << signedKeysType << " _lower = _keys;\n" + " " << signedKeysType << " _mid;\n" + " " << signedKeysType << " _upper = " << CAST(signedKeysType) << + " (_keys + _klen - 1);\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = " << CAST(signedKeysType) << + " (_lower + ((_upper-_lower) >> 1));\n" + " if ( " << GET_WIDE_KEY() << " < " << K() << "[_mid] )\n" + " _upper = " << CAST(signedKeysType) << " (_mid - 1);\n" + " else if ( " << GET_WIDE_KEY() << " > " << K() << "[_mid] )\n" + " _lower = " << CAST(signedKeysType) << " (_mid + 1);\n" + " else {\n" + " _trans += " << CAST(transType) << " (_mid - _keys);\n" + " goto _match;\n" + " }\n" + " }\n" + " _keys += " << CAST(keysType) << " _klen;\n" + " _trans += " << CAST(transType) << " _klen;\n" + " }\n" + "\n" + " _klen = " << RL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " " << signedKeysType << " _lower = _keys;\n" + " " << signedKeysType << " _mid;\n" + " " << signedKeysType << " _upper = " << CAST(signedKeysType) << + " (_keys + (_klen<<1) - 2);\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = " << CAST(signedKeysType) << + " (_lower + (((_upper-_lower) >> 1) & ~1));\n" + " if ( " << GET_WIDE_KEY() << " < " << K() << "[_mid] )\n" + " _upper = " << CAST(signedKeysType) << " (_mid - 2);\n" + " else if ( " << GET_WIDE_KEY() << " > " << K() << "[_mid+1] )\n" + " _lower = " << CAST(signedKeysType) << " (_mid + 2);\n" + " else {\n" + " _trans += " << CAST(transType) << "((_mid - _keys)>>1);\n" + " goto _match;\n" + " }\n" + " }\n" + " _trans += " << CAST(transType) << " _klen;\n" + " }\n" + "\n"; +} + +void CSharpTabCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n" + " _klen = " << CL() << "[" << vCS() << "];\n" + " _keys = " << CAST(keysType) << " ("<< CO() << "[" << vCS() << "]*2);\n" + " if ( _klen > 0 ) {\n" + " " << signedKeysType << " _lower = _keys;\n" + " " << signedKeysType << " _mid;\n" + " " << signedKeysType << " _upper = " << CAST(signedKeysType) << + " (_keys + (_klen<<1) - 2);\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = " << CAST(signedKeysType) << + " (_lower + (((_upper-_lower) >> 1) & ~1));\n" + " if ( " << GET_WIDE_KEY() << " < " << CK() << "[_mid] )\n" + " _upper = " << CAST(signedKeysType) << " (_mid - 2);\n" + " else if ( " << GET_WIDE_KEY() << " > " << CK() << "[_mid+1] )\n" + " _lower = " << CAST(signedKeysType) << " (_mid + 2);\n" + " else {\n" + " switch ( " << C() << "[" << CO() << "[" << vCS() << "]" + " + ((_mid - _keys)>>1)] ) {\n"; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId << ": {\n"; + out << TABS(2) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << + " break;\n" + " }\n"; + } + + SWITCH_DEFAULT(); + + out << + " }\n" + " break;\n" + " }\n" + " }\n" + " }\n" + "\n"; +} + +void CSharpTabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " {\n" + " " << klenType << " _klen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " " << transType << " _trans;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " int _acts;\n" + " int _nacts;\n"; + } + + out << + " " << keysType << " _keys;\n" + "\n"; +// " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + + if ( !noEnd ) { + testEofUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n" + " goto _test_eof;\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + out << "_resume:\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" + vCS() + "]" << ";\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + out << "_match:\n"; + + if ( useIndicies ) + out << " _trans = " << CAST(transType) << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] == 0 )\n" + " goto _again;\n" + "\n" + " _acts = " << TA() << "[_trans]" << ";\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 )\n {\n" + " switch ( " << A() << "[_acts++] )\n {\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "]" << ";\n" + " _nacts = " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n" + " goto _out;\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " )\n" + " goto _resume;\n"; + } + else { + out << + " " << P() << " += 1;\n" + " goto _resume;\n"; + } + + if ( testEofUsed ) + out << " _test_eof: {}\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << CAST(transType) << " (" << ET() << + "[" << vCS() << "] - 1);\n" + " goto _eof_trans;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " int __acts = " << + EA() << "[" << vCS() << "]" << ";\n" + " int __nacts = " << + A() << "[__acts++];\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( " << A() << "[__acts++] ) {\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + if ( outLabelUsed ) + out << " _out: {}\n"; + + out << " }\n"; +} + +void CSharpTabCodeGen::initVarTypes() +{ + int klenMax = MAX(MAX(redFsm->maxCondLen, redFsm->maxRangeLen), + redFsm->maxSingleLen); + int keysMax = MAX(MAX(redFsm->maxKeyOffset, klenMax), + redFsm->maxCondOffset); + int transMax = MAX(MAX(redFsm->maxIndex+1, redFsm->maxIndexOffset), keysMax); + transMax = MAX(transMax, klenMax); + transType = ARRAY_TYPE(transMax); + klenType = ARRAY_TYPE(klenMax); + keysType = ARRAY_TYPE(keysMax); + signedKeysType = ARRAY_TYPE(keysMax, true); +} diff --git a/contrib/tools/ragel6/cstable.h b/contrib/tools/ragel6/cstable.h new file mode 100644 index 0000000000..a8206fc9ab --- /dev/null +++ b/contrib/tools/ragel6/cstable.h @@ -0,0 +1,100 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _TABCODEGEN_H +#define _TABCODEGEN_H + +#include <iostream> +#include "cscodegen.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; + +/* + * TabCodeGen + */ +class CSharpTabCodeGen : virtual public CSharpFsmCodeGen, public CSharpCodeGen +{ +public: + CSharpTabCodeGen( ostream &out ) : CSharpFsmCodeGen(out), CSharpCodeGen(out) {} + virtual ~CSharpTabCodeGen() { } + virtual void writeData(); + virtual void writeExec(); + +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + std::ostream &COND_KEYS(); + std::ostream &COND_SPACES(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &COND_OFFSETS(); + std::ostream &KEY_OFFSETS(); + std::ostream &INDEX_OFFSETS(); + std::ostream &COND_LENS(); + std::ostream &SINGLE_LENS(); + std::ostream &RANGE_LENS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + std::ostream &TRANS_TARGS_WI(); + std::ostream &TRANS_ACTIONS_WI(); + + void LOCATE_TRANS(); + + void COND_TRANSLATE(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void calcIndexSize(); + + void initVarTypes(); + string klenType; + string keysType; + string signedKeysType; + string transType; +}; + +#endif diff --git a/contrib/tools/ragel6/dotcodegen.cpp b/contrib/tools/ragel6/dotcodegen.cpp new file mode 100644 index 0000000000..7a83c46c0d --- /dev/null +++ b/contrib/tools/ragel6/dotcodegen.cpp @@ -0,0 +1,320 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "dotcodegen.h" +#include "gendata.h" + +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + +/* Override this so that write statement processing is ignored */ +bool GraphvizDotGen::writeStatement( InputLoc &, int, char ** ) +{ + return false; +} + +std::ostream &GraphvizDotGen::KEY( Key key ) +{ + if ( displayPrintables && key.isPrintable() ) { + // Output values as characters, ensuring we escape the quote (") character + char cVal = (char) key.getVal(); + switch ( cVal ) { + case '"': case '\\': + out << "'\\" << cVal << "'"; + break; + case '\a': + out << "'\\\\a'"; + break; + case '\b': + out << "'\\\\b'"; + break; + case '\t': + out << "'\\\\t'"; + break; + case '\n': + out << "'\\\\n'"; + break; + case '\v': + out << "'\\\\v'"; + break; + case '\f': + out << "'\\\\f'"; + break; + case '\r': + out << "'\\\\r'"; + break; + case ' ': + out << "SP"; + break; + default: + out << "'" << cVal << "'"; + break; + } + } + else { + if ( keyOps->isSigned ) + out << key.getVal(); + else + out << (unsigned long) key.getVal(); + } + + return out; +} + +std::ostream &GraphvizDotGen::TRANS_ACTION( RedStateAp *fromState, RedTransAp *trans ) +{ + int n = 0; + RedAction *actions[3]; + + if ( fromState->fromStateAction != 0 ) + actions[n++] = fromState->fromStateAction; + if ( trans->action != 0 ) + actions[n++] = trans->action; + if ( trans->targ != 0 && trans->targ->toStateAction != 0 ) + actions[n++] = trans->targ->toStateAction; + + if ( n > 0 ) + out << " / "; + + /* Loop the existing actions and write out what's there. */ + for ( int a = 0; a < n; a++ ) { + for ( GenActionTable::Iter actIt = actions[a]->key.first(); actIt.lte(); actIt++ ) { + GenAction *action = actIt->value; + out << action->nameOrLoc(); + if ( a < n-1 || !actIt.last() ) + out << ", "; + } + } + return out; +} + +std::ostream &GraphvizDotGen::ACTION( RedAction *action ) +{ + /* The action. */ + out << " / "; + for ( GenActionTable::Iter actIt = action->key.first(); actIt.lte(); actIt++ ) { + GenAction *action = actIt->value; + if ( action->name != 0 ) + out << action->name; + else + out << action->loc.line << ":" << action->loc.col; + if ( !actIt.last() ) + out << ", "; + } + return out; +} + +std::ostream &GraphvizDotGen::ONCHAR( Key lowKey, Key highKey ) +{ + GenCondSpace *condSpace; + if ( lowKey > keyOps->maxKey && (condSpace=findCondSpace(lowKey, highKey) ) ) { + Key values = ( lowKey - condSpace->baseKey ) / keyOps->alphSize(); + + lowKey = keyOps->minKey + + (lowKey - condSpace->baseKey - keyOps->alphSize() * values.getVal()); + highKey = keyOps->minKey + + (highKey - condSpace->baseKey - keyOps->alphSize() * values.getVal()); + KEY( lowKey ); + if ( lowKey != highKey ) { + out << ".."; + KEY( highKey ); + } + out << "("; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + bool set = values & (1 << csi.pos()); + if ( !set ) + out << "!"; + out << (*csi)->nameOrLoc(); + if ( !csi.last() ) + out << ", "; + } + out << ")"; + } + else { + /* Output the key. Possibly a range. */ + KEY( lowKey ); + if ( highKey != lowKey ) { + out << ".."; + KEY( highKey ); + } + } + return out; +} + +void GraphvizDotGen::writeTransList( RedStateAp *state ) +{ + /* Build the set of unique transitions out of this state. */ + RedTransSet stTransSet; + for ( RedTransList::Iter tel = state->outRange; tel.lte(); tel++ ) { + /* If we haven't seen the transitions before, the move forward + * emitting all the transitions on the same character. */ + if ( stTransSet.insert( tel->value ) ) { + /* Write out the from and to states. */ + out << "\t" << state->id << " -> "; + + if ( tel->value->targ == 0 ) + out << "err_" << state->id; + else + out << tel->value->targ->id; + + /* Begin the label. */ + out << " [ label = \""; + ONCHAR( tel->lowKey, tel->highKey ); + + /* Walk the transition list, finding the same. */ + for ( RedTransList::Iter mtel = tel.next(); mtel.lte(); mtel++ ) { + if ( mtel->value == tel->value ) { + out << ", "; + ONCHAR( mtel->lowKey, mtel->highKey ); + } + } + + /* Write the action and close the transition. */ + TRANS_ACTION( state, tel->value ); + out << "\" ];\n"; + } + } + + /* Write the default transition. */ + if ( state->defTrans != 0 ) { + /* Write out the from and to states. */ + out << "\t" << state->id << " -> "; + + if ( state->defTrans->targ == 0 ) + out << "err_" << state->id; + else + out << state->defTrans->targ->id; + + /* Begin the label. */ + out << " [ label = \"DEF"; + + /* Write the action and close the transition. */ + TRANS_ACTION( state, state->defTrans ); + out << "\" ];\n"; + } +} + +void GraphvizDotGen::writeDotFile( ) +{ + out << + "digraph " << fsmName << " {\n" + " rankdir=LR;\n"; + + /* Define the psuedo states. Transitions will be done after the states + * have been defined as either final or not final. */ + out << " node [ shape = point ];\n"; + + if ( redFsm->startState != 0 ) + out << " ENTRY;\n"; + + /* Psuedo states for entry points in the entry map. */ + for ( EntryIdVect::Iter en = entryPointIds; en.lte(); en++ ) { + RedStateAp *state = allStates + *en; + out << " en_" << state->id << ";\n"; + } + + /* Psuedo states for final states with eof actions. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 && st->eofTrans->action != 0 ) + out << " eof_" << st->id << ";\n"; + if ( st->eofAction != 0 ) + out << " eof_" << st->id << ";\n"; + } + + out << " node [ shape = circle, height = 0.2 ];\n"; + + /* Psuedo states for states whose default actions go to error. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + bool needsErr = false; + if ( st->defTrans != 0 && st->defTrans->targ == 0 ) + needsErr = true; + else { + for ( RedTransList::Iter tel = st->outRange; tel.lte(); tel++ ) { + if ( tel->value->targ == 0 ) { + needsErr = true; + break; + } + } + } + + if ( needsErr ) + out << " err_" << st->id << " [ label=\"\"];\n"; + } + + /* Attributes common to all nodes, plus double circle for final states. */ + out << " node [ fixedsize = true, height = 0.65, shape = doublecircle ];\n"; + + /* List Final states. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->isFinal ) + out << " " << st->id << ";\n"; + } + + /* List transitions. */ + out << " node [ shape = circle ];\n"; + + /* Walk the states. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + writeTransList( st ); + + /* Transitions into the start state. */ + if ( redFsm->startState != 0 ) + out << " ENTRY -> " << redFsm->startState->id << " [ label = \"IN\" ];\n"; + + /* Transitions into the entry points. */ + for ( EntryIdVect::Iter en = entryPointIds; en.lte(); en++ ) { + RedStateAp *state = allStates + *en; + char *name = entryPointNames[en.pos()]; + out << " en_" << state->id << " -> " << state->id << + " [ label = \"" << name << "\" ];\n"; + } + + /* Out action transitions. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 && st->eofTrans->action != 0 ) { + out << " " << st->id << " -> eof_" << + st->id << " [ label = \"EOF"; + ACTION( st->eofTrans->action ) << "\" ];\n"; + } + if ( st->eofAction != 0 ) { + out << " " << st->id << " -> eof_" << + st->id << " [ label = \"EOF"; + ACTION( st->eofAction ) << "\" ];\n"; + } + } + + out << + "}\n"; +} + +void GraphvizDotGen::finishRagelDef() +{ + /* For dot file generation we want to pick default transitions. */ + redFsm->chooseDefaultSpan(); +} diff --git a/contrib/tools/ragel6/dotcodegen.h b/contrib/tools/ragel6/dotcodegen.h new file mode 100644 index 0000000000..97dd3afa41 --- /dev/null +++ b/contrib/tools/ragel6/dotcodegen.h @@ -0,0 +1,48 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GVDOTGEN_H +#define _GVDOTGEN_H + +#include <iostream> +#include "gendata.h" + +class GraphvizDotGen : public CodeGenData +{ +public: + GraphvizDotGen( ostream &out ) : CodeGenData(out) { } + + /* Print an fsm to out stream. */ + void writeTransList( RedStateAp *state ); + void writeDotFile( ); + + virtual void finishRagelDef(); + virtual bool writeStatement( InputLoc &, int, char ** ); + +private: + /* Writing labels and actions. */ + std::ostream &ONCHAR( Key lowKey, Key highKey ); + std::ostream &TRANS_ACTION( RedStateAp *fromState, RedTransAp *trans ); + std::ostream &ACTION( RedAction *action ); + std::ostream &KEY( Key key ); +}; + +#endif diff --git a/contrib/tools/ragel6/fsmap.cpp b/contrib/tools/ragel6/fsmap.cpp new file mode 100644 index 0000000000..0a9879987b --- /dev/null +++ b/contrib/tools/ragel6/fsmap.cpp @@ -0,0 +1,879 @@ +/* + * Copyright 2002-2004 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "fsmgraph.h" +#include <iostream> +using std::cerr; +using std::endl; + +CondData *condData = 0; +KeyOps *keyOps = 0; + +/* Insert an action into an action table. */ +void ActionTable::setAction( int ordering, Action *action ) +{ + /* Multi-insert in case specific instances of an action appear in a + * transition more than once. */ + insertMulti( ordering, action ); +} + +/* Set all the action from another action table in this table. */ +void ActionTable::setActions( const ActionTable &other ) +{ + for ( ActionTable::Iter action = other; action.lte(); action++ ) + insertMulti( action->key, action->value ); +} + +void ActionTable::setActions( int *orderings, Action **actions, int nActs ) +{ + for ( int a = 0; a < nActs; a++ ) + insertMulti( orderings[a], actions[a] ); +} + +bool ActionTable::hasAction( Action *action ) +{ + for ( int a = 0; a < length(); a++ ) { + if ( data[a].value == action ) + return true; + } + return false; +} + +/* Insert an action into an action table. */ +void LmActionTable::setAction( int ordering, LongestMatchPart *action ) +{ + /* Multi-insert in case specific instances of an action appear in a + * transition more than once. */ + insertMulti( ordering, action ); +} + +/* Set all the action from another action table in this table. */ +void LmActionTable::setActions( const LmActionTable &other ) +{ + for ( LmActionTable::Iter action = other; action.lte(); action++ ) + insertMulti( action->key, action->value ); +} + +void ErrActionTable::setAction( int ordering, Action *action, int transferPoint ) +{ + insertMulti( ErrActionTableEl( action, ordering, transferPoint ) ); +} + +void ErrActionTable::setActions( const ErrActionTable &other ) +{ + for ( ErrActionTable::Iter act = other; act.lte(); act++ ) + insertMulti( ErrActionTableEl( act->action, act->ordering, act->transferPoint ) ); +} + +/* Insert a priority into this priority table. Looks out for priorities on + * duplicate keys. */ +void PriorTable::setPrior( int ordering, PriorDesc *desc ) +{ + PriorEl *lastHit = 0; + PriorEl *insed = insert( PriorEl(ordering, desc), &lastHit ); + if ( insed == 0 ) { + /* This already has a priority on the same key as desc. Overwrite the + * priority if the ordering is larger (later in time). */ + if ( ordering >= lastHit->ordering ) + *lastHit = PriorEl( ordering, desc ); + } +} + +/* Set all the priorities from a priorTable in this table. */ +void PriorTable::setPriors( const PriorTable &other ) +{ + /* Loop src priorities once to overwrite duplicates. */ + PriorTable::Iter priorIt = other; + for ( ; priorIt.lte(); priorIt++ ) + setPrior( priorIt->ordering, priorIt->desc ); +} + +/* Set the priority of starting transitions. Isolates the start state so it has + * no other entry points, then sets the priorities of all the transitions out + * of the start state. If the start state is final, then the outPrior of the + * start state is also set. The idea is that a machine that accepts the null + * string can still specify the starting trans prior for when it accepts the + * null word. */ +void FsmAp::startFsmPrior( int ordering, PriorDesc *prior ) +{ + /* Make sure the start state has no other entry points. */ + isolateStartState(); + + /* Walk all transitions out of the start state. */ + for ( TransList::Iter trans = startState->outList; trans.lte(); trans++ ) { + if ( trans->toState != 0 ) + trans->priorTable.setPrior( ordering, prior ); + } + + /* If the new start state is final then set the out priority. This follows + * the same convention as setting start action in the out action table of + * a final start state. */ + if ( startState->stateBits & STB_ISFINAL ) + startState->outPriorTable.setPrior( ordering, prior ); +} + +/* Set the priority of all transitions in a graph. Walks all transition lists + * and all def transitions. */ +void FsmAp::allTransPrior( int ordering, PriorDesc *prior ) +{ + /* Walk the list of all states. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + /* Walk the out list of the state. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + if ( trans->toState != 0 ) + trans->priorTable.setPrior( ordering, prior ); + } + } +} + +/* Set the priority of all transitions that go into a final state. Note that if + * any entry states are final, we will not be setting the priority of any + * transitions that may go into those states in the future. The graph does not + * support pending in transitions in the same way pending out transitions are + * supported. */ +void FsmAp::finishFsmPrior( int ordering, PriorDesc *prior ) +{ + /* Walk all final states. */ + for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) { + /* Walk all in transitions of the final state. */ + for ( TransInList::Iter trans = (*state)->inList; trans.lte(); trans++ ) + trans->priorTable.setPrior( ordering, prior ); + } +} + +/* Set the priority of any future out transitions that may be made going out of + * this state machine. */ +void FsmAp::leaveFsmPrior( int ordering, PriorDesc *prior ) +{ + /* Set priority in all final states. */ + for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) + (*state)->outPriorTable.setPrior( ordering, prior ); +} + + +/* Set actions to execute on starting transitions. Isolates the start state + * so it has no other entry points, then adds to the transition functions + * of all the transitions out of the start state. If the start state is final, + * then the func is also added to the start state's out func list. The idea is + * that a machine that accepts the null string can execute a start func when it + * matches the null word, which can only be done when leaving the start/final + * state. */ +void FsmAp::startFsmAction( int ordering, Action *action ) +{ + /* Make sure the start state has no other entry points. */ + isolateStartState(); + + /* Walk the start state's transitions, setting functions. */ + for ( TransList::Iter trans = startState->outList; trans.lte(); trans++ ) { + if ( trans->toState != 0 ) + trans->actionTable.setAction( ordering, action ); + } + + /* If start state is final then add the action to the out action table. + * This means that when the null string is accepted the start action will + * not be bypassed. */ + if ( startState->stateBits & STB_ISFINAL ) + startState->outActionTable.setAction( ordering, action ); +} + +/* Set functions to execute on all transitions. Walks the out lists of all + * states. */ +void FsmAp::allTransAction( int ordering, Action *action ) +{ + /* Walk all states. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + /* Walk the out list of the state. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + if ( trans->toState != 0 ) + trans->actionTable.setAction( ordering, action ); + } + } +} + +/* Specify functions to execute upon entering final states. If the start state + * is final we can't really specify a function to execute upon entering that + * final state the first time. So function really means whenever entering a + * final state from within the same fsm. */ +void FsmAp::finishFsmAction( int ordering, Action *action ) +{ + /* Walk all final states. */ + for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) { + /* Walk the final state's in list. */ + for ( TransInList::Iter trans = (*state)->inList; trans.lte(); trans++ ) + trans->actionTable.setAction( ordering, action ); + } +} + +/* Add functions to any future out transitions that may be made going out of + * this state machine. */ +void FsmAp::leaveFsmAction( int ordering, Action *action ) +{ + /* Insert the action in the outActionTable of all final states. */ + for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) + (*state)->outActionTable.setAction( ordering, action ); +} + +/* Add functions to the longest match action table for constructing scanners. */ +void FsmAp::longMatchAction( int ordering, LongestMatchPart *lmPart ) +{ + /* Walk all final states. */ + for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) { + /* Walk the final state's in list. */ + for ( TransInList::Iter trans = (*state)->inList; trans.lte(); trans++ ) + trans->lmActionTable.setAction( ordering, lmPart ); + } +} + +void FsmAp::fillGaps( StateAp *state ) +{ + if ( state->outList.length() == 0 ) { + /* Add the range on the lower and upper bound. */ + attachNewTrans( state, 0, keyOps->minKey, keyOps->maxKey ); + } + else { + TransList srcList; + srcList.transfer( state->outList ); + + /* Check for a gap at the beginning. */ + TransList::Iter trans = srcList, next; + if ( keyOps->minKey < trans->lowKey ) { + /* Make the high key and append. */ + Key highKey = trans->lowKey; + highKey.decrement(); + + attachNewTrans( state, 0, keyOps->minKey, highKey ); + } + + /* Write the transition. */ + next = trans.next(); + state->outList.append( trans ); + + /* Keep the last high end. */ + Key lastHigh = trans->highKey; + + /* Loop each source range. */ + for ( trans = next; trans.lte(); trans = next ) { + /* Make the next key following the last range. */ + Key nextKey = lastHigh; + nextKey.increment(); + + /* Check for a gap from last up to here. */ + if ( nextKey < trans->lowKey ) { + /* Make the high end of the range that fills the gap. */ + Key highKey = trans->lowKey; + highKey.decrement(); + + attachNewTrans( state, 0, nextKey, highKey ); + } + + /* Reduce the transition. If it reduced to anything then add it. */ + next = trans.next(); + state->outList.append( trans ); + + /* Keep the last high end. */ + lastHigh = trans->highKey; + } + + /* Now check for a gap on the end to fill. */ + if ( lastHigh < keyOps->maxKey ) { + /* Get a copy of the default. */ + lastHigh.increment(); + + attachNewTrans( state, 0, lastHigh, keyOps->maxKey ); + } + } +} + +void FsmAp::setErrorActions( StateAp *state, const ActionTable &other ) +{ + /* Fill any gaps in the out list with an error transition. */ + fillGaps( state ); + + /* Set error transitions in the transitions that go to error. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + if ( trans->toState == 0 ) + trans->actionTable.setActions( other ); + } +} + +void FsmAp::setErrorAction( StateAp *state, int ordering, Action *action ) +{ + /* Fill any gaps in the out list with an error transition. */ + fillGaps( state ); + + /* Set error transitions in the transitions that go to error. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + if ( trans->toState == 0 ) + trans->actionTable.setAction( ordering, action ); + } +} + + +/* Give a target state for error transitions. */ +void FsmAp::setErrorTarget( StateAp *state, StateAp *target, int *orderings, + Action **actions, int nActs ) +{ + /* Fill any gaps in the out list with an error transition. */ + fillGaps( state ); + + /* Set error target in the transitions that go to error. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + if ( trans->toState == 0 ) { + /* The trans goes to error, redirect it. */ + redirectErrorTrans( trans->fromState, target, trans ); + trans->actionTable.setActions( orderings, actions, nActs ); + } + } +} + +void FsmAp::transferOutActions( StateAp *state ) +{ + for ( ActionTable::Iter act = state->outActionTable; act.lte(); act++ ) + state->eofActionTable.setAction( act->key, act->value ); + state->outActionTable.empty(); +} + +void FsmAp::transferErrorActions( StateAp *state, int transferPoint ) +{ + for ( int i = 0; i < state->errActionTable.length(); ) { + ErrActionTableEl *act = state->errActionTable.data + i; + if ( act->transferPoint == transferPoint ) { + /* Transfer the error action and remove it. */ + setErrorAction( state, act->ordering, act->action ); + if ( ! state->isFinState() ) + state->eofActionTable.setAction( act->ordering, act->action ); + state->errActionTable.vremove( i ); + } + else { + /* Not transfering and deleting, skip over the item. */ + i += 1; + } + } +} + +/* Set error actions in the start state. */ +void FsmAp::startErrorAction( int ordering, Action *action, int transferPoint ) +{ + /* Make sure the start state has no other entry points. */ + isolateStartState(); + + /* Add the actions. */ + startState->errActionTable.setAction( ordering, action, transferPoint ); +} + +/* Set error actions in all states where there is a transition out. */ +void FsmAp::allErrorAction( int ordering, Action *action, int transferPoint ) +{ + /* Insert actions in the error action table of all states. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) + state->errActionTable.setAction( ordering, action, transferPoint ); +} + +/* Set error actions in final states. */ +void FsmAp::finalErrorAction( int ordering, Action *action, int transferPoint ) +{ + /* Add the action to the error table of final states. */ + for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) + (*state)->errActionTable.setAction( ordering, action, transferPoint ); +} + +void FsmAp::notStartErrorAction( int ordering, Action *action, int transferPoint ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( state != startState ) + state->errActionTable.setAction( ordering, action, transferPoint ); + } +} + +void FsmAp::notFinalErrorAction( int ordering, Action *action, int transferPoint ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( ! state->isFinState() ) + state->errActionTable.setAction( ordering, action, transferPoint ); + } +} + +/* Set error actions in the states that have transitions into a final state. */ +void FsmAp::middleErrorAction( int ordering, Action *action, int transferPoint ) +{ + /* Isolate the start state in case it is reachable from in inside the + * machine, in which case we don't want it set. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( state != startState && ! state->isFinState() ) + state->errActionTable.setAction( ordering, action, transferPoint ); + } +} + +/* Set EOF actions in the start state. */ +void FsmAp::startEOFAction( int ordering, Action *action ) +{ + /* Make sure the start state has no other entry points. */ + isolateStartState(); + + /* Add the actions. */ + startState->eofActionTable.setAction( ordering, action ); +} + +/* Set EOF actions in all states where there is a transition out. */ +void FsmAp::allEOFAction( int ordering, Action *action ) +{ + /* Insert actions in the EOF action table of all states. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) + state->eofActionTable.setAction( ordering, action ); +} + +/* Set EOF actions in final states. */ +void FsmAp::finalEOFAction( int ordering, Action *action ) +{ + /* Add the action to the error table of final states. */ + for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) + (*state)->eofActionTable.setAction( ordering, action ); +} + +void FsmAp::notStartEOFAction( int ordering, Action *action ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( state != startState ) + state->eofActionTable.setAction( ordering, action ); + } +} + +void FsmAp::notFinalEOFAction( int ordering, Action *action ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( ! state->isFinState() ) + state->eofActionTable.setAction( ordering, action ); + } +} + +/* Set EOF actions in the states that have transitions into a final state. */ +void FsmAp::middleEOFAction( int ordering, Action *action ) +{ + /* Set the actions in all states that are not the start state and not final. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( state != startState && ! state->isFinState() ) + state->eofActionTable.setAction( ordering, action ); + } +} + +/* + * Set To State Actions. + */ + +/* Set to state actions in the start state. */ +void FsmAp::startToStateAction( int ordering, Action *action ) +{ + /* Make sure the start state has no other entry points. */ + isolateStartState(); + startState->toStateActionTable.setAction( ordering, action ); +} + +/* Set to state actions in all states. */ +void FsmAp::allToStateAction( int ordering, Action *action ) +{ + /* Insert the action on all states. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) + state->toStateActionTable.setAction( ordering, action ); +} + +/* Set to state actions in final states. */ +void FsmAp::finalToStateAction( int ordering, Action *action ) +{ + /* Add the action to the error table of final states. */ + for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) + (*state)->toStateActionTable.setAction( ordering, action ); +} + +void FsmAp::notStartToStateAction( int ordering, Action *action ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( state != startState ) + state->toStateActionTable.setAction( ordering, action ); + } +} + +void FsmAp::notFinalToStateAction( int ordering, Action *action ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( ! state->isFinState() ) + state->toStateActionTable.setAction( ordering, action ); + } +} + +/* Set to state actions in states that are not final and not the start state. */ +void FsmAp::middleToStateAction( int ordering, Action *action ) +{ + /* Set the action in all states that are not the start state and not final. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( state != startState && ! state->isFinState() ) + state->toStateActionTable.setAction( ordering, action ); + } +} + +/* + * Set From State Actions. + */ + +void FsmAp::startFromStateAction( int ordering, Action *action ) +{ + /* Make sure the start state has no other entry points. */ + isolateStartState(); + startState->fromStateActionTable.setAction( ordering, action ); +} + +void FsmAp::allFromStateAction( int ordering, Action *action ) +{ + /* Insert the action on all states. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) + state->fromStateActionTable.setAction( ordering, action ); +} + +void FsmAp::finalFromStateAction( int ordering, Action *action ) +{ + /* Add the action to the error table of final states. */ + for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) + (*state)->fromStateActionTable.setAction( ordering, action ); +} + +void FsmAp::notStartFromStateAction( int ordering, Action *action ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( state != startState ) + state->fromStateActionTable.setAction( ordering, action ); + } +} + +void FsmAp::notFinalFromStateAction( int ordering, Action *action ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( ! state->isFinState() ) + state->fromStateActionTable.setAction( ordering, action ); + } +} + +void FsmAp::middleFromStateAction( int ordering, Action *action ) +{ + /* Set the action in all states that are not the start state and not final. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + if ( state != startState && ! state->isFinState() ) + state->fromStateActionTable.setAction( ordering, action ); + } +} + +/* Shift the function ordering of the start transitions to start + * at fromOrder and increase in units of 1. Useful before staring. + * Returns the maximum number of order numbers used. */ +int FsmAp::shiftStartActionOrder( int fromOrder ) +{ + int maxUsed = 0; + + /* Walk the start state's transitions, shifting function ordering. */ + for ( TransList::Iter trans = startState->outList; trans.lte(); trans++ ) { + /* Walk the function data for the transition and set the keys to + * increasing values starting at fromOrder. */ + int curFromOrder = fromOrder; + ActionTable::Iter action = trans->actionTable; + for ( ; action.lte(); action++ ) + action->key = curFromOrder++; + + /* Keep track of the max number of orders used. */ + if ( curFromOrder - fromOrder > maxUsed ) + maxUsed = curFromOrder - fromOrder; + } + + return maxUsed; +} + +/* Remove all priorities. */ +void FsmAp::clearAllPriorities() +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + /* Clear out priority data. */ + state->outPriorTable.empty(); + + /* Clear transition data from the out transitions. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) + trans->priorTable.empty(); + } +} + +/* Zeros out the function ordering keys. This may be called before minimization + * when it is known that no more fsm operations are going to be done. This + * will achieve greater reduction as states will not be separated on the basis + * of function ordering. */ +void FsmAp::nullActionKeys( ) +{ + /* For each state... */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + /* Walk the transitions for the state. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + /* Walk the action table for the transition. */ + for ( ActionTable::Iter action = trans->actionTable; + action.lte(); action++ ) + action->key = 0; + + /* Walk the action table for the transition. */ + for ( LmActionTable::Iter action = trans->lmActionTable; + action.lte(); action++ ) + action->key = 0; + } + + /* Null the action keys of the to state action table. */ + for ( ActionTable::Iter action = state->toStateActionTable; + action.lte(); action++ ) + action->key = 0; + + /* Null the action keys of the from state action table. */ + for ( ActionTable::Iter action = state->fromStateActionTable; + action.lte(); action++ ) + action->key = 0; + + /* Null the action keys of the out transtions. */ + for ( ActionTable::Iter action = state->outActionTable; + action.lte(); action++ ) + action->key = 0; + + /* Null the action keys of the error action table. */ + for ( ErrActionTable::Iter action = state->errActionTable; + action.lte(); action++ ) + action->ordering = 0; + + /* Null the action keys eof action table. */ + for ( ActionTable::Iter action = state->eofActionTable; + action.lte(); action++ ) + action->key = 0; + } +} + +/* Walk the list of states and verify that non final states do not have out + * data, that all stateBits are cleared, and that there are no states with + * zero foreign in transitions. */ +void FsmAp::verifyStates() +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + /* Non final states should not have leaving data. */ + if ( ! (state->stateBits & STB_ISFINAL) ) { + assert( state->outActionTable.length() == 0 ); + assert( state->outCondSet.length() == 0 ); + assert( state->outPriorTable.length() == 0 ); + } + + /* Data used in algorithms should be cleared. */ + assert( (state->stateBits & STB_BOTH) == 0 ); + assert( state->foreignInTrans > 0 ); + } +} + +/* Compare two transitions according to their relative priority. Since the + * base transition has no priority associated with it, the default is to + * return equal. */ +int FsmAp::comparePrior( const PriorTable &priorTable1, const PriorTable &priorTable2 ) +{ + /* Looking for differing priorities on same keys. Need to concurrently + * scan the priority lists. */ + PriorTable::Iter pd1 = priorTable1; + PriorTable::Iter pd2 = priorTable2; + while ( pd1.lte() && pd2.lte() ) { + /* Check keys. */ + if ( pd1->desc->key < pd2->desc->key ) + pd1.increment(); + else if ( pd1->desc->key > pd2->desc->key ) + pd2.increment(); + /* Keys are the same, check priorities. */ + else if ( pd1->desc->priority < pd2->desc->priority ) + return -1; + else if ( pd1->desc->priority > pd2->desc->priority ) + return 1; + else { + /* Keys and priorities are equal, advance both. */ + pd1.increment(); + pd2.increment(); + } + } + + /* No differing priorities on the same key. */ + return 0; +} + +/* Compares two transitions according to priority and functions. Pointers + * should not be null. Does not consider to state or from state. Compare two + * transitions according to the data contained in the transitions. Data means + * any properties added to user transitions that may differentiate them. Since + * the base transition has no data, the default is to return equal. */ +int FsmAp::compareTransData( TransAp *trans1, TransAp *trans2 ) +{ + /* Compare the prior table. */ + int cmpRes = CmpPriorTable::compare( trans1->priorTable, + trans2->priorTable ); + if ( cmpRes != 0 ) + return cmpRes; + + /* Compare longest match action tables. */ + cmpRes = CmpLmActionTable::compare(trans1->lmActionTable, + trans2->lmActionTable); + if ( cmpRes != 0 ) + return cmpRes; + + /* Compare action tables. */ + return CmpActionTable::compare(trans1->actionTable, + trans2->actionTable); +} + +/* Callback invoked when another trans (or possibly this) is added into this + * transition during the merging process. Draw in any properties of srcTrans + * into this transition. AddInTrans is called when a new transitions is made + * that will be a duplicate of another transition or a combination of several + * other transitions. AddInTrans will be called for each transition that the + * new transition is to represent. */ +void FsmAp::addInTrans( TransAp *destTrans, TransAp *srcTrans ) +{ + /* Protect against adding in from ourselves. */ + if ( srcTrans == destTrans ) { + /* Adding in ourselves, need to make a copy of the source transitions. + * The priorities are not copied in as that would have no effect. */ + destTrans->lmActionTable.setActions( LmActionTable(srcTrans->lmActionTable) ); + destTrans->actionTable.setActions( ActionTable(srcTrans->actionTable) ); + } + else { + /* Not a copy of ourself, get the functions and priorities. */ + destTrans->lmActionTable.setActions( srcTrans->lmActionTable ); + destTrans->actionTable.setActions( srcTrans->actionTable ); + destTrans->priorTable.setPriors( srcTrans->priorTable ); + } +} + +/* Compare the properties of states that are embedded by users. Compares out + * priorities, out transitions, to, from, out, error and eof action tables. */ +int FsmAp::compareStateData( const StateAp *state1, const StateAp *state2 ) +{ + /* Compare the out priority table. */ + int cmpRes = CmpPriorTable:: + compare( state1->outPriorTable, state2->outPriorTable ); + if ( cmpRes != 0 ) + return cmpRes; + + /* Test to state action tables. */ + cmpRes = CmpActionTable::compare( state1->toStateActionTable, + state2->toStateActionTable ); + if ( cmpRes != 0 ) + return cmpRes; + + /* Test from state action tables. */ + cmpRes = CmpActionTable::compare( state1->fromStateActionTable, + state2->fromStateActionTable ); + if ( cmpRes != 0 ) + return cmpRes; + + /* Test out action tables. */ + cmpRes = CmpActionTable::compare( state1->outActionTable, + state2->outActionTable ); + if ( cmpRes != 0 ) + return cmpRes; + + /* Test out condition sets. */ + cmpRes = CmpOutCondSet::compare( state1->outCondSet, + state2->outCondSet ); + if ( cmpRes != 0 ) + return cmpRes; + + /* Test out error action tables. */ + cmpRes = CmpErrActionTable::compare( state1->errActionTable, + state2->errActionTable ); + if ( cmpRes != 0 ) + return cmpRes; + + /* Test eof action tables. */ + return CmpActionTable::compare( state1->eofActionTable, + state2->eofActionTable ); +} + + +/* Invoked when a state looses its final state status and the leaving + * transition embedding data should be deleted. */ +void FsmAp::clearOutData( StateAp *state ) +{ + /* Kill the out actions and priorities. */ + state->outActionTable.empty(); + state->outCondSet.empty(); + state->outPriorTable.empty(); +} + +bool FsmAp::hasOutData( StateAp *state ) +{ + return ( state->outActionTable.length() > 0 || + state->outCondSet.length() > 0 || + state->outPriorTable.length() > 0 ); +} + +/* + * Setting Conditions. + */ + +void logNewExpansion( Expansion *exp ); +void logCondSpace( CondSpace *condSpace ); + +CondSpace *FsmAp::addCondSpace( const CondSet &condSet ) +{ + CondSpace *condSpace = condData->condSpaceMap.find( condSet ); + if ( condSpace == 0 ) { + /* Do we have enough keyspace left? */ + Size availableSpace = condData->lastCondKey.availableSpace(); + Size neededSpace = (1 << condSet.length() ) * keyOps->alphSize(); + if ( neededSpace > availableSpace ) + throw FsmConstructFail( FsmConstructFail::CondNoKeySpace ); + + Key baseKey = condData->lastCondKey; + baseKey.increment(); + condData->lastCondKey += (1 << condSet.length() ) * keyOps->alphSize(); + + condSpace = new CondSpace( condSet ); + condSpace->baseKey = baseKey; + condData->condSpaceMap.insert( condSpace ); + + #ifdef LOG_CONDS + cerr << "adding new condition space" << endl; + cerr << " condition set: "; + logCondSpace( condSpace ); + cerr << endl; + cerr << " baseKey: " << baseKey.getVal() << endl; + #endif + } + return condSpace; +} + +void FsmAp::startFsmCondition( Action *condAction, bool sense ) +{ + /* Make sure the start state has no other entry points. */ + isolateStartState(); + embedCondition( startState, condAction, sense ); +} + +void FsmAp::allTransCondition( Action *condAction, bool sense ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) + embedCondition( state, condAction, sense ); +} + +void FsmAp::leaveFsmCondition( Action *condAction, bool sense ) +{ + for ( StateSet::Iter state = finStateSet; state.lte(); state++ ) + (*state)->outCondSet.insert( OutCond( condAction, sense ) ); +} diff --git a/contrib/tools/ragel6/fsmattach.cpp b/contrib/tools/ragel6/fsmattach.cpp new file mode 100644 index 0000000000..b0911f3112 --- /dev/null +++ b/contrib/tools/ragel6/fsmattach.cpp @@ -0,0 +1,425 @@ +/* + * Copyright 2001 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <string.h> +#include <assert.h> +#include "fsmgraph.h" + +#include <iostream> +using namespace std; + +/* Insert a transition into an inlist. The head must be supplied. */ +void FsmAp::attachToInList( StateAp *from, StateAp *to, + TransAp *&head, TransAp *trans ) +{ + trans->ilnext = head; + trans->ilprev = 0; + + /* If in trans list is not empty, set the head->prev to trans. */ + if ( head != 0 ) + head->ilprev = trans; + + /* Now insert ourselves at the front of the list. */ + head = trans; + + /* Keep track of foreign transitions for from and to. */ + if ( from != to ) { + if ( misfitAccounting ) { + /* If the number of foreign in transitions is about to go up to 1 then + * move it from the misfit list to the main list. */ + if ( to->foreignInTrans == 0 ) + stateList.append( misfitList.detach( to ) ); + } + + to->foreignInTrans += 1; + } +}; + +/* Detach a transition from an inlist. The head of the inlist must be supplied. */ +void FsmAp::detachFromInList( StateAp *from, StateAp *to, + TransAp *&head, TransAp *trans ) +{ + /* Detach in the inTransList. */ + if ( trans->ilprev == 0 ) + head = trans->ilnext; + else + trans->ilprev->ilnext = trans->ilnext; + + if ( trans->ilnext != 0 ) + trans->ilnext->ilprev = trans->ilprev; + + /* Keep track of foreign transitions for from and to. */ + if ( from != to ) { + to->foreignInTrans -= 1; + + if ( misfitAccounting ) { + /* If the number of foreign in transitions goes down to 0 then move it + * from the main list to the misfit list. */ + if ( to->foreignInTrans == 0 ) + misfitList.append( stateList.detach( to ) ); + } + } +} + +/* Attach states on the default transition, range list or on out/in list key. + * First makes a new transition. If there is already a transition out from + * fromState on the default, then will assertion fail. */ +TransAp *FsmAp::attachNewTrans( StateAp *from, StateAp *to, Key lowKey, Key highKey ) +{ + /* Make the new transition. */ + TransAp *retVal = new TransAp(); + + /* The transition is now attached. Remember the parties involved. */ + retVal->fromState = from; + retVal->toState = to; + + /* Make the entry in the out list for the transitions. */ + from->outList.append( retVal ); + + /* Set the the keys of the new trans. */ + retVal->lowKey = lowKey; + retVal->highKey = highKey; + + /* Attach using inList as the head pointer. */ + if ( to != 0 ) + attachToInList( from, to, to->inList.head, retVal ); + + return retVal; +} + +/* Attach for range lists or for the default transition. This attach should + * be used when a transition already is allocated and must be attached to a + * target state. Does not handle adding the transition into the out list. */ +void FsmAp::attachTrans( StateAp *from, StateAp *to, TransAp *trans ) +{ + assert( trans->fromState == 0 && trans->toState == 0 ); + trans->fromState = from; + trans->toState = to; + + if ( to != 0 ) { + /* Attach using the inList pointer as the head pointer. */ + attachToInList( from, to, to->inList.head, trans ); + } +} + +/* Redirect a transition away from error and towards some state. This is just + * like attachTrans except it requires fromState to be set and does not touch + * it. */ +void FsmAp::redirectErrorTrans( StateAp *from, StateAp *to, TransAp *trans ) +{ + assert( trans->fromState != 0 && trans->toState == 0 ); + trans->toState = to; + + if ( to != 0 ) { + /* Attach using the inList pointer as the head pointer. */ + attachToInList( from, to, to->inList.head, trans ); + } +} + +/* Detach for out/in lists or for default transition. */ +void FsmAp::detachTrans( StateAp *from, StateAp *to, TransAp *trans ) +{ + assert( trans->fromState == from && trans->toState == to ); + trans->fromState = 0; + trans->toState = 0; + + if ( to != 0 ) { + /* Detach using to's inList pointer as the head. */ + detachFromInList( from, to, to->inList.head, trans ); + } +} + + +/* Detach a state from the graph. Detaches and deletes transitions in and out + * of the state. Empties inList and outList. Removes the state from the final + * state set. A detached state becomes useless and should be deleted. */ +void FsmAp::detachState( StateAp *state ) +{ + /* Detach the in transitions from the inList list of transitions. */ + while ( state->inList.head != 0 ) { + /* Get pointers to the trans and the state. */ + TransAp *trans = state->inList.head; + StateAp *fromState = trans->fromState; + + /* Detach the transitions from the source state. */ + detachTrans( fromState, state, trans ); + + /* Ok to delete the transition. */ + fromState->outList.detach( trans ); + delete trans; + } + + /* Remove the entry points in on the machine. */ + while ( state->entryIds.length() > 0 ) + unsetEntry( state->entryIds[0], state ); + + /* Detach out range transitions. */ + for ( TransList::Iter trans = state->outList; trans.lte(); ) { + TransList::Iter next = trans.next(); + detachTrans( state, trans->toState, trans ); + delete trans; + trans = next; + } + + /* Delete all of the out range pointers. */ + state->outList.abandon(); + + /* Unset final stateness before detaching from graph. */ + if ( state->stateBits & STB_ISFINAL ) + finStateSet.remove( state ); +} + + +/* Duplicate a transition. Makes a new transition that is attached to the same + * dest as srcTrans. The new transition has functions and priority taken from + * srcTrans. Used for merging a transition in to a free spot. The trans can + * just be dropped in. It does not conflict with an existing trans and need + * not be crossed. Returns the new transition. */ +TransAp *FsmAp::dupTrans( StateAp *from, TransAp *srcTrans ) +{ + /* Make a new transition. */ + TransAp *newTrans = new TransAp(); + + /* We can attach the transition, one does not exist. */ + attachTrans( from, srcTrans->toState, newTrans ); + + /* Call the user callback to add in the original source transition. */ + addInTrans( newTrans, srcTrans ); + + return newTrans; +} + +/* In crossing, src trans and dest trans both go to existing states. Make one + * state from the sets of states that src and dest trans go to. */ +TransAp *FsmAp::fsmAttachStates( MergeData &md, StateAp *from, + TransAp *destTrans, TransAp *srcTrans ) +{ + /* The priorities are equal. We must merge the transitions. Does the + * existing trans go to the state we are to attach to? ie, are we to + * simply double up the transition? */ + StateAp *toState = srcTrans->toState; + StateAp *existingState = destTrans->toState; + + if ( existingState == toState ) { + /* The transition is a double up to the same state. Copy the src + * trans into itself. We don't need to merge in the from out trans + * data, that was done already. */ + addInTrans( destTrans, srcTrans ); + } + else { + /* The trans is not a double up. Dest trans cannot be the same as src + * trans. Set up the state set. */ + StateSet stateSet; + + /* We go to all the states the existing trans goes to, plus... */ + if ( existingState->stateDictEl == 0 ) + stateSet.insert( existingState ); + else + stateSet.insert( existingState->stateDictEl->stateSet ); + + /* ... all the states that we have been told to go to. */ + if ( toState->stateDictEl == 0 ) + stateSet.insert( toState ); + else + stateSet.insert( toState->stateDictEl->stateSet ); + + /* Look for the state. If it is not there already, make it. */ + StateDictEl *lastFound; + if ( md.stateDict.insert( stateSet, &lastFound ) ) { + /* Make a new state representing the combination of states in + * stateSet. It gets added to the fill list. This means that we + * need to fill in it's transitions sometime in the future. We + * don't do that now (ie, do not recurse). */ + StateAp *combinState = addState(); + + /* Link up the dict element and the state. */ + lastFound->targState = combinState; + combinState->stateDictEl = lastFound; + + /* Add to the fill list. */ + md.fillListAppend( combinState ); + } + + /* Get the state insertted/deleted. */ + StateAp *targ = lastFound->targState; + + /* Detach the state from existing state. */ + detachTrans( from, existingState, destTrans ); + + /* Re-attach to the new target. */ + attachTrans( from, targ, destTrans ); + + /* Add in src trans to the existing transition that we redirected to + * the new state. We don't need to merge in the from out trans data, + * that was done already. */ + addInTrans( destTrans, srcTrans ); + } + + return destTrans; +} + +/* Two transitions are to be crossed, handle the possibility of either going + * to the error state. */ +TransAp *FsmAp::mergeTrans( MergeData &md, StateAp *from, + TransAp *destTrans, TransAp *srcTrans ) +{ + TransAp *retTrans = 0; + if ( destTrans->toState == 0 && srcTrans->toState == 0 ) { + /* Error added into error. */ + addInTrans( destTrans, srcTrans ); + retTrans = destTrans; + } + else if ( destTrans->toState == 0 && srcTrans->toState != 0 ) { + /* Non error added into error we need to detach and reattach, */ + detachTrans( from, destTrans->toState, destTrans ); + attachTrans( from, srcTrans->toState, destTrans ); + addInTrans( destTrans, srcTrans ); + retTrans = destTrans; + } + else if ( srcTrans->toState == 0 ) { + /* Dest goes somewhere but src doesn't, just add it it in. */ + addInTrans( destTrans, srcTrans ); + retTrans = destTrans; + } + else { + /* Both go somewhere, run the actual cross. */ + retTrans = fsmAttachStates( md, from, destTrans, srcTrans ); + } + + return retTrans; +} + +/* Find the trans with the higher priority. If src is lower priority then dest then + * src is ignored. If src is higher priority than dest, then src overwrites dest. If + * the priorities are equal, then they are merged. */ +TransAp *FsmAp::crossTransitions( MergeData &md, StateAp *from, + TransAp *destTrans, TransAp *srcTrans ) +{ + TransAp *retTrans; + + /* Compare the priority of the dest and src transitions. */ + int compareRes = comparePrior( destTrans->priorTable, srcTrans->priorTable ); + if ( compareRes < 0 ) { + /* Src trans has a higher priority than dest, src overwrites dest. + * Detach dest and return a copy of src. */ + detachTrans( from, destTrans->toState, destTrans ); + retTrans = dupTrans( from, srcTrans ); + } + else if ( compareRes > 0 ) { + /* The dest trans has a higher priority, use dest. */ + retTrans = destTrans; + } + else { + /* Src trans and dest trans have the same priority, they must be merged. */ + retTrans = mergeTrans( md, from, destTrans, srcTrans ); + } + + /* Return the transition that resulted from the cross. */ + return retTrans; +} + +/* Copy the transitions in srcList to the outlist of dest. The srcList should + * not be the outList of dest, otherwise you would be copying the contents of + * srcList into itself as it's iterated: bad news. */ +void FsmAp::outTransCopy( MergeData &md, StateAp *dest, TransAp *srcList ) +{ + /* The destination list. */ + TransList destList; + + /* Set up an iterator to stop at breaks. */ + PairIter<TransAp> outPair( dest->outList.head, srcList ); + for ( ; !outPair.end(); outPair++ ) { + switch ( outPair.userState ) { + case RangeInS1: { + /* The pair iter is the authority on the keys. It may have needed + * to break the dest range. */ + TransAp *destTrans = outPair.s1Tel.trans; + destTrans->lowKey = outPair.s1Tel.lowKey; + destTrans->highKey = outPair.s1Tel.highKey; + destList.append( destTrans ); + break; + } + case RangeInS2: { + /* Src range may get crossed with dest's default transition. */ + TransAp *newTrans = dupTrans( dest, outPair.s2Tel.trans ); + + /* Set up the transition's keys and append to the dest list. */ + newTrans->lowKey = outPair.s2Tel.lowKey; + newTrans->highKey = outPair.s2Tel.highKey; + destList.append( newTrans ); + break; + } + case RangeOverlap: { + /* Exact overlap, cross them. */ + TransAp *newTrans = crossTransitions( md, dest, + outPair.s1Tel.trans, outPair.s2Tel.trans ); + + /* Set up the transition's keys and append to the dest list. */ + newTrans->lowKey = outPair.s1Tel.lowKey; + newTrans->highKey = outPair.s1Tel.highKey; + destList.append( newTrans ); + break; + } + case BreakS1: { + /* Since we are always writing to the dest trans, the dest needs + * to be copied when it is broken. The copy goes into the first + * half of the break to "break it off". */ + outPair.s1Tel.trans = dupTrans( dest, outPair.s1Tel.trans ); + break; + } + case BreakS2: + break; + } + } + + /* Abandon the old outList and transfer destList into it. */ + dest->outList.transfer( destList ); +} + + +/* Move all the transitions that go into src so that they go into dest. */ +void FsmAp::inTransMove( StateAp *dest, StateAp *src ) +{ + /* Do not try to move in trans to and from the same state. */ + assert( dest != src ); + + /* If src is the start state, dest becomes the start state. */ + if ( src == startState ) { + unsetStartState(); + setStartState( dest ); + } + + /* For each entry point into, create an entry point into dest, when the + * state is detached, the entry points to src will be removed. */ + for ( EntryIdSet::Iter enId = src->entryIds; enId.lte(); enId++ ) + changeEntry( *enId, dest, src ); + + /* Move the transitions in inList. */ + while ( src->inList.head != 0 ) { + /* Get trans and from state. */ + TransAp *trans = src->inList.head; + StateAp *fromState = trans->fromState; + + /* Detach from src, reattach to dest. */ + detachTrans( fromState, src, trans ); + attachTrans( fromState, dest, trans ); + } +} diff --git a/contrib/tools/ragel6/fsmbase.cpp b/contrib/tools/ragel6/fsmbase.cpp new file mode 100644 index 0000000000..87c7f0e15b --- /dev/null +++ b/contrib/tools/ragel6/fsmbase.cpp @@ -0,0 +1,602 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <string.h> +#include <assert.h> +#include "fsmgraph.h" + +/* Simple singly linked list append routine for the fill list. The new state + * goes to the end of the list. */ +void MergeData::fillListAppend( StateAp *state ) +{ + state->alg.next = 0; + + if ( stfillHead == 0 ) { + /* List is empty, state becomes head and tail. */ + stfillHead = state; + stfillTail = state; + } + else { + /* List is not empty, state goes after last element. */ + stfillTail->alg.next = state; + stfillTail = state; + } +} + +/* Graph constructor. */ +FsmAp::FsmAp() +: + /* No start state. */ + startState(0), + errState(0), + + /* Misfit accounting is a switch, turned on only at specific times. It + * controls what happens when states have no way in from the outside + * world.. */ + misfitAccounting(false) +{ +} + +/* Copy all graph data including transitions. */ +FsmAp::FsmAp( const FsmAp &graph ) +: + /* Lists start empty. Will be filled by copy. */ + stateList(), + misfitList(), + + /* Copy in the entry points, + * pointers will be resolved later. */ + entryPoints(graph.entryPoints), + startState(graph.startState), + errState(0), + + /* Will be filled by copy. */ + finStateSet(), + + /* Misfit accounting is only on during merging. */ + misfitAccounting(false) +{ + /* Create the states and record their map in the original state. */ + StateList::Iter origState = graph.stateList; + for ( ; origState.lte(); origState++ ) { + /* Make the new state. */ + StateAp *newState = new StateAp( *origState ); + + /* Add the state to the list. */ + stateList.append( newState ); + + /* Set the mapsTo item of the old state. */ + origState->alg.stateMap = newState; + } + + /* Derefernce all the state maps. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + /* The points to the original in the src machine. The taget's duplicate + * is in the statemap. */ + StateAp *toState = trans->toState != 0 ? trans->toState->alg.stateMap : 0; + + /* Attach The transition to the duplicate. */ + trans->toState = 0; + attachTrans( state, toState, trans ); + } + + /* Fix the eofTarg, if set. */ + if ( state->eofTarget != 0 ) + state->eofTarget = state->eofTarget->alg.stateMap; + } + + /* Fix the state pointers in the entry points array. */ + EntryMapEl *eel = entryPoints.data; + for ( int e = 0; e < entryPoints.length(); e++, eel++ ) { + /* Get the duplicate of the state. */ + eel->value = eel->value->alg.stateMap; + + /* Foreign in transitions must be built up when duping machines so + * increment it here. */ + eel->value->foreignInTrans += 1; + } + + /* Fix the start state pointer and the new start state's count of in + * transiions. */ + startState = startState->alg.stateMap; + startState->foreignInTrans += 1; + + /* Build the final state set. */ + StateSet::Iter st = graph.finStateSet; + for ( ; st.lte(); st++ ) + finStateSet.insert((*st)->alg.stateMap); +} + +/* Deletes all transition data then deletes each state. */ +FsmAp::~FsmAp() +{ + /* Delete all the transitions. */ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + /* Iterate the out transitions, deleting them. */ + state->outList.empty(); + } + + /* Delete all the states. */ + stateList.empty(); +} + +/* Set a state final. The state has its isFinState set to true and the state + * is added to the finStateSet. */ +void FsmAp::setFinState( StateAp *state ) +{ + /* Is it already a fin state. */ + if ( state->stateBits & STB_ISFINAL ) + return; + + state->stateBits |= STB_ISFINAL; + finStateSet.insert( state ); +} + +/* Set a state non-final. The has its isFinState flag set false and the state + * is removed from the final state set. */ +void FsmAp::unsetFinState( StateAp *state ) +{ + /* Is it already a non-final state? */ + if ( ! (state->stateBits & STB_ISFINAL) ) + return; + + /* When a state looses its final state status it must relinquish all the + * properties that are allowed only for final states. */ + clearOutData( state ); + + state->stateBits &= ~ STB_ISFINAL; + finStateSet.remove( state ); +} + +/* Set and unset a state as the start state. */ +void FsmAp::setStartState( StateAp *state ) +{ + /* Sould change from unset to set. */ + assert( startState == 0 ); + startState = state; + + if ( misfitAccounting ) { + /* If the number of foreign in transitions is about to go up to 1 then + * take it off the misfit list and put it on the head list. */ + if ( state->foreignInTrans == 0 ) + stateList.append( misfitList.detach( state ) ); + } + + /* Up the foreign in transitions to the state. */ + state->foreignInTrans += 1; +} + +void FsmAp::unsetStartState() +{ + /* Should change from set to unset. */ + assert( startState != 0 ); + + /* Decrement the entry's count of foreign entries. */ + startState->foreignInTrans -= 1; + + if ( misfitAccounting ) { + /* If the number of foreign in transitions just went down to 0 then take + * it off the main list and put it on the misfit list. */ + if ( startState->foreignInTrans == 0 ) + misfitList.append( stateList.detach( startState ) ); + } + + startState = 0; +} + +/* Associate an id with a state. Makes the state a named entry point. Has no + * effect if the entry point is already mapped to the state. */ +void FsmAp::setEntry( int id, StateAp *state ) +{ + /* Insert the id into the state. If the state is already labelled with id, + * nothing to do. */ + if ( state->entryIds.insert( id ) ) { + /* Insert the entry and assert that it succeeds. */ + entryPoints.insertMulti( id, state ); + + if ( misfitAccounting ) { + /* If the number of foreign in transitions is about to go up to 1 then + * take it off the misfit list and put it on the head list. */ + if ( state->foreignInTrans == 0 ) + stateList.append( misfitList.detach( state ) ); + } + + /* Up the foreign in transitions to the state. */ + state->foreignInTrans += 1; + } +} + +/* Remove the association of an id with a state. The state looses it's entry + * point status. Assumes that the id is indeed mapped to state. */ +void FsmAp::unsetEntry( int id, StateAp *state ) +{ + /* Find the entry point in on id. */ + EntryMapEl *enLow = 0, *enHigh = 0; + entryPoints.findMulti( id, enLow, enHigh ); + while ( enLow->value != state ) + enLow += 1; + + /* Remove the record from the map. */ + entryPoints.remove( enLow ); + + /* Remove the state's sense of the link. */ + state->entryIds.remove( id ); + state->foreignInTrans -= 1; + if ( misfitAccounting ) { + /* If the number of foreign in transitions just went down to 0 then take + * it off the main list and put it on the misfit list. */ + if ( state->foreignInTrans == 0 ) + misfitList.append( stateList.detach( state ) ); + } +} + +/* Remove all association of an id with states. Assumes that the id is indeed + * mapped to a state. */ +void FsmAp::unsetEntry( int id ) +{ + /* Find the entry point in on id. */ + EntryMapEl *enLow = 0, *enHigh = 0; + entryPoints.findMulti( id, enLow, enHigh ); + for ( EntryMapEl *mel = enLow; mel <= enHigh; mel++ ) { + /* Remove the state's sense of the link. */ + mel->value->entryIds.remove( id ); + mel->value->foreignInTrans -= 1; + if ( misfitAccounting ) { + /* If the number of foreign in transitions just went down to 0 + * then take it off the main list and put it on the misfit list. */ + if ( mel->value->foreignInTrans == 0 ) + misfitList.append( stateList.detach( mel->value ) ); + } + } + + /* Remove the records from the entry points map. */ + entryPoints.removeMulti( enLow, enHigh ); +} + + +void FsmAp::changeEntry( int id, StateAp *to, StateAp *from ) +{ + /* Find the entry in the entry map. */ + EntryMapEl *enLow = 0, *enHigh = 0; + entryPoints.findMulti( id, enLow, enHigh ); + while ( enLow->value != from ) + enLow += 1; + + /* Change it to the new target. */ + enLow->value = to; + + /* Remove from's sense of the link. */ + from->entryIds.remove( id ); + from->foreignInTrans -= 1; + if ( misfitAccounting ) { + /* If the number of foreign in transitions just went down to 0 then take + * it off the main list and put it on the misfit list. */ + if ( from->foreignInTrans == 0 ) + misfitList.append( stateList.detach( from ) ); + } + + /* Add to's sense of the link. */ + if ( to->entryIds.insert( id ) != 0 ) { + if ( misfitAccounting ) { + /* If the number of foreign in transitions is about to go up to 1 then + * take it off the misfit list and put it on the head list. */ + if ( to->foreignInTrans == 0 ) + stateList.append( misfitList.detach( to ) ); + } + + /* Up the foreign in transitions to the state. */ + to->foreignInTrans += 1; + } +} + + +/* Clear all entry points from a machine. */ +void FsmAp::unsetAllEntryPoints() +{ + for ( EntryMap::Iter en = entryPoints; en.lte(); en++ ) { + /* Kill all the state's entry points at once. */ + if ( en->value->entryIds.length() > 0 ) { + en->value->foreignInTrans -= en->value->entryIds.length(); + + if ( misfitAccounting ) { + /* If the number of foreign in transitions just went down to 0 + * then take it off the main list and put it on the misfit + * list. */ + if ( en->value->foreignInTrans == 0 ) + misfitList.append( stateList.detach( en->value ) ); + } + + /* Clear the set of ids out all at once. */ + en->value->entryIds.empty(); + } + } + + /* Now clear out the entry map all at once. */ + entryPoints.empty(); +} + +/* Assigning an epsilon transition into final states. */ +void FsmAp::epsilonTrans( int id ) +{ + for ( StateSet::Iter fs = finStateSet; fs.lte(); fs++ ) + (*fs)->epsilonTrans.append( id ); +} + +/* Mark all states reachable from state. Traverses transitions forward. Used + * for removing states that have no path into them. */ +void FsmAp::markReachableFromHere( StateAp *state ) +{ + /* Base case: return; */ + if ( state->stateBits & STB_ISMARKED ) + return; + + /* Set this state as processed. We are going to visit all states that this + * state has a transition to. */ + state->stateBits |= STB_ISMARKED; + + /* Recurse on all out transitions. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + if ( trans->toState != 0 ) + markReachableFromHere( trans->toState ); + } +} + +void FsmAp::markReachableFromHereStopFinal( StateAp *state ) +{ + /* Base case: return; */ + if ( state->stateBits & STB_ISMARKED ) + return; + + /* Set this state as processed. We are going to visit all states that this + * state has a transition to. */ + state->stateBits |= STB_ISMARKED; + + /* Recurse on all out transitions. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + StateAp *toState = trans->toState; + if ( toState != 0 && !toState->isFinState() ) + markReachableFromHereStopFinal( toState ); + } +} + +/* Mark all states reachable from state. Traverse transitions backwards. Used + * for removing dead end paths in graphs. */ +void FsmAp::markReachableFromHereReverse( StateAp *state ) +{ + /* Base case: return; */ + if ( state->stateBits & STB_ISMARKED ) + return; + + /* Set this state as processed. We are going to visit all states with + * transitions into this state. */ + state->stateBits |= STB_ISMARKED; + + /* Recurse on all items in transitions. */ + for ( TransInList::Iter trans = state->inList; trans.lte(); trans++ ) + markReachableFromHereReverse( trans->fromState ); +} + +/* Determine if there are any entry points into a start state other than the + * start state. Setting starting transitions requires that the start state be + * isolated. In most cases a start state will already be isolated. */ +bool FsmAp::isStartStateIsolated() +{ + /* If there are any in transitions then the state is not isolated. */ + if ( startState->inList.head != 0 ) + return false; + + /* If there are any entry points then isolated. */ + if ( startState->entryIds.length() > 0 ) + return false; + + return true; +} + +/* Bring in other's entry points. Assumes others states are going to be + * copied into this machine. */ +void FsmAp::copyInEntryPoints( FsmAp *other ) +{ + /* Use insert multi because names are not unique. */ + for ( EntryMap::Iter en = other->entryPoints; en.lte(); en++ ) + entryPoints.insertMulti( en->key, en->value ); +} + + +void FsmAp::unsetAllFinStates() +{ + for ( StateSet::Iter st = finStateSet; st.lte(); st++ ) + (*st)->stateBits &= ~ STB_ISFINAL; + finStateSet.empty(); +} + +void FsmAp::setFinBits( int finStateBits ) +{ + for ( int s = 0; s < finStateSet.length(); s++ ) + finStateSet.data[s]->stateBits |= finStateBits; +} + + +/* Tests the integrity of the transition lists and the fromStates. */ +void FsmAp::verifyIntegrity() +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) { + /* Walk the out transitions and assert fromState is correct. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) + assert( trans->fromState == state ); + + /* Walk the inlist and assert toState is correct. */ + for ( TransInList::Iter trans = state->inList; trans.lte(); trans++ ) + assert( trans->toState == state ); + } +} + +void FsmAp::verifyReachability() +{ + /* Mark all the states that can be reached + * through the set of entry points. */ + markReachableFromHere( startState ); + for ( EntryMap::Iter en = entryPoints; en.lte(); en++ ) + markReachableFromHere( en->value ); + + /* Check that everything got marked. */ + for ( StateList::Iter st = stateList; st.lte(); st++ ) { + /* Assert it got marked and then clear the mark. */ + assert( st->stateBits & STB_ISMARKED ); + st->stateBits &= ~ STB_ISMARKED; + } +} + +void FsmAp::verifyNoDeadEndStates() +{ + /* Mark all states that have paths to the final states. */ + for ( StateSet::Iter pst = finStateSet; pst.lte(); pst++ ) + markReachableFromHereReverse( *pst ); + + /* Start state gets honorary marking. Must be done AFTER recursive call. */ + startState->stateBits |= STB_ISMARKED; + + /* Make sure everything got marked. */ + for ( StateList::Iter st = stateList; st.lte(); st++ ) { + /* Assert the state got marked and unmark it. */ + assert( st->stateBits & STB_ISMARKED ); + st->stateBits &= ~ STB_ISMARKED; + } +} + +void FsmAp::depthFirstOrdering( StateAp *state ) +{ + /* Nothing to do if the state is already on the list. */ + if ( state->stateBits & STB_ONLIST ) + return; + + /* Doing depth first, put state on the list. */ + state->stateBits |= STB_ONLIST; + stateList.append( state ); + + /* Recurse on everything ranges. */ + for ( TransList::Iter tel = state->outList; tel.lte(); tel++ ) { + if ( tel->toState != 0 ) + depthFirstOrdering( tel->toState ); + } +} + +/* Ordering states by transition connections. */ +void FsmAp::depthFirstOrdering() +{ + /* Init on state list flags. */ + for ( StateList::Iter st = stateList; st.lte(); st++ ) + st->stateBits &= ~STB_ONLIST; + + /* Clear out the state list, we will rebuild it. */ + int stateListLen = stateList.length(); + stateList.abandon(); + + /* Add back to the state list from the start state and all other entry + * points. */ + if ( errState != 0 ) + depthFirstOrdering( errState ); + depthFirstOrdering( startState ); + for ( EntryMap::Iter en = entryPoints; en.lte(); en++ ) + depthFirstOrdering( en->value ); + + /* Make sure we put everything back on. */ + assert( stateListLen == stateList.length() ); +} + +/* Stable sort the states by final state status. */ +void FsmAp::sortStatesByFinal() +{ + /* Move forward through the list and throw final states onto the end. */ + StateAp *state = 0; + StateAp *next = stateList.head; + StateAp *last = stateList.tail; + while ( state != last ) { + /* Move forward and load up the next. */ + state = next; + next = state->next; + + /* Throw to the end? */ + if ( state->isFinState() ) { + stateList.detach( state ); + stateList.append( state ); + } + } +} + +void FsmAp::setStateNumbers( int base ) +{ + for ( StateList::Iter state = stateList; state.lte(); state++ ) + state->alg.stateNum = base++; +} + + +bool FsmAp::checkErrTrans( StateAp *state, TransAp *trans ) +{ + /* Might go directly to error state. */ + if ( trans->toState == 0 ) + return true; + + if ( trans->prev == 0 ) { + /* If this is the first transition. */ + if ( keyOps->minKey < trans->lowKey ) + return true; + } + else { + /* Not the first transition. Compare against the prev. */ + TransAp *prev = trans->prev; + Key nextKey = prev->highKey; + nextKey.increment(); + if ( nextKey < trans->lowKey ) + return true; + } + return false; +} + +bool FsmAp::checkErrTransFinish( StateAp *state ) +{ + /* Check if there are any ranges already. */ + if ( state->outList.length() == 0 ) + return true; + else { + /* Get the last and check for a gap on the end. */ + TransAp *last = state->outList.tail; + if ( last->highKey < keyOps->maxKey ) + return true; + } + return 0; +} + +bool FsmAp::hasErrorTrans() +{ + bool result; + for ( StateList::Iter st = stateList; st.lte(); st++ ) { + for ( TransList::Iter tr = st->outList; tr.lte(); tr++ ) { + result = checkErrTrans( st, tr ); + if ( result ) + return true; + } + result = checkErrTransFinish( st ); + if ( result ) + return true; + } + return false; +} diff --git a/contrib/tools/ragel6/fsmgraph.cpp b/contrib/tools/ragel6/fsmgraph.cpp new file mode 100644 index 0000000000..fecc4a52b9 --- /dev/null +++ b/contrib/tools/ragel6/fsmgraph.cpp @@ -0,0 +1,1434 @@ +/* + * Copyright 2001, 2002, 2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <assert.h> +#include <iostream> + +#include "fsmgraph.h" +#include "mergesort.h" +#include "parsedata.h" + +using std::cerr; +using std::endl; + +/* Make a new state. The new state will be put on the graph's + * list of state. The new state can be created final or non final. */ +StateAp *FsmAp::addState() +{ + /* Make the new state to return. */ + StateAp *state = new StateAp(); + + if ( misfitAccounting ) { + /* Create the new state on the misfit list. All states are created + * with no foreign in transitions. */ + misfitList.append( state ); + } + else { + /* Create the new state. */ + stateList.append( state ); + } + + return state; +} + +/* Construct an FSM that is the concatenation of an array of characters. A new + * machine will be made that has len+1 states with one transition between each + * state for each integer in str. IsSigned determines if the integers are to + * be considered as signed or unsigned ints. */ +void FsmAp::concatFsm( Key *str, int len ) +{ + /* Make the first state and set it as the start state. */ + StateAp *last = addState(); + setStartState( last ); + + /* Attach subsequent states. */ + for ( int i = 0; i < len; i++ ) { + StateAp *newState = addState(); + attachNewTrans( last, newState, str[i], str[i] ); + last = newState; + } + + /* Make the last state the final state. */ + setFinState( last ); +} + +/* Case insensitive version of concatFsm. */ +void FsmAp::concatFsmCI( Key *str, int len ) +{ + /* Make the first state and set it as the start state. */ + StateAp *last = addState(); + setStartState( last ); + + /* Attach subsequent states. */ + for ( int i = 0; i < len; i++ ) { + StateAp *newState = addState(); + + KeySet keySet; + if ( str[i].isLower() ) + keySet.insert( str[i].toUpper() ); + if ( str[i].isUpper() ) + keySet.insert( str[i].toLower() ); + keySet.insert( str[i] ); + + for ( int i = 0; i < keySet.length(); i++ ) + attachNewTrans( last, newState, keySet[i], keySet[i] ); + + last = newState; + } + + /* Make the last state the final state. */ + setFinState( last ); +} + +/* Construct a machine that matches one character. A new machine will be made + * that has two states with a single transition between the states. IsSigned + * determines if the integers are to be considered as signed or unsigned ints. */ +void FsmAp::concatFsm( Key chr ) +{ + /* Two states first start, second final. */ + setStartState( addState() ); + + StateAp *end = addState(); + setFinState( end ); + + /* Attach on the character. */ + attachNewTrans( startState, end, chr, chr ); +} + +/* Construct a machine that matches any character in set. A new machine will + * be made that has two states and len transitions between the them. The set + * should be ordered correctly accroding to KeyOps and should not contain + * any duplicates. */ +void FsmAp::orFsm( Key *set, int len ) +{ + /* Two states first start, second final. */ + setStartState( addState() ); + + StateAp *end = addState(); + setFinState( end ); + + for ( int i = 1; i < len; i++ ) + assert( set[i-1] < set[i] ); + + /* Attach on all the integers in the given string of ints. */ + for ( int i = 0; i < len; i++ ) + attachNewTrans( startState, end, set[i], set[i] ); +} + +/* Construct a machine that matches a range of characters. A new machine will + * be made with two states and a range transition between them. The range will + * match any characters from low to high inclusive. Low should be less than or + * equal to high otherwise undefined behaviour results. IsSigned determines + * if the integers are to be considered as signed or unsigned ints. */ +void FsmAp::rangeFsm( Key low, Key high ) +{ + /* Two states first start, second final. */ + setStartState( addState() ); + + StateAp *end = addState(); + setFinState( end ); + + /* Attach using the range of characters. */ + attachNewTrans( startState, end, low, high ); +} + +/* Construct a machine that a repeated range of characters. */ +void FsmAp::rangeStarFsm( Key low, Key high) +{ + /* One state which is final and is the start state. */ + setStartState( addState() ); + setFinState( startState ); + + /* Attach start to start using range of characters. */ + attachNewTrans( startState, startState, low, high ); +} + +/* Construct a machine that matches the empty string. A new machine will be + * made with only one state. The new state will be both a start and final + * state. IsSigned determines if the machine has a signed or unsigned + * alphabet. Fsm operations must be done on machines with the same alphabet + * signedness. */ +void FsmAp::lambdaFsm( ) +{ + /* Give it one state with no transitions making it + * the start state and final state. */ + setStartState( addState() ); + setFinState( startState ); +} + +/* Construct a machine that matches nothing at all. A new machine will be + * made with only one state. It will not be final. */ +void FsmAp::emptyFsm( ) +{ + /* Give it one state with no transitions making it + * the start state and final state. */ + setStartState( addState() ); +} + +void FsmAp::transferOutData( StateAp *destState, StateAp *srcState ) +{ + for ( TransList::Iter trans = destState->outList; trans.lte(); trans++ ) { + if ( trans->toState != 0 ) { + /* Get the actions data from the outActionTable. */ + trans->actionTable.setActions( srcState->outActionTable ); + + /* Get the priorities from the outPriorTable. */ + trans->priorTable.setPriors( srcState->outPriorTable ); + } + } +} + +/* Kleene star operator. Makes this machine the kleene star of itself. Any + * transitions made going out of the machine and back into itself will be + * notified that they are leaving transitions by having the leavingFromState + * callback invoked. */ +void FsmAp::starOp( ) +{ + /* For the merging process. */ + MergeData md; + + /* Turn on misfit accounting to possibly catch the old start state. */ + setMisfitAccounting( true ); + + /* Create the new new start state. It will be set final after the merging + * of the final states with the start state is complete. */ + StateAp *prevStartState = startState; + unsetStartState(); + setStartState( addState() ); + + /* Merge the new start state with the old one to isolate it. */ + mergeStates( md, startState, prevStartState ); + + /* Merge the start state into all final states. Except the start state on + * the first pass. If the start state is set final we will be doubling up + * its transitions, which will get transfered to any final states that + * follow it in the final state set. This will be determined by the order + * of items in the final state set. To prevent this we just merge with the + * start on a second pass. */ + for ( StateSet::Iter st = finStateSet; st.lte(); st++ ) { + if ( *st != startState ) + mergeStatesLeaving( md, *st, startState ); + } + + /* Now it is safe to merge the start state with itself (provided it + * is set final). */ + if ( startState->isFinState() ) + mergeStatesLeaving( md, startState, startState ); + + /* Now ensure the new start state is a final state. */ + setFinState( startState ); + + /* Fill in any states that were newed up as combinations of others. */ + fillInStates( md ); + + /* Remove the misfits and turn off misfit accounting. */ + removeMisfits(); + setMisfitAccounting( false ); +} + +void FsmAp::repeatOp( int times ) +{ + /* Must be 1 and up. 0 produces null machine and requires deleting this. */ + assert( times > 0 ); + + /* A repeat of one does absolutely nothing. */ + if ( times == 1 ) + return; + + /* Make a machine to make copies from. */ + FsmAp *copyFrom = new FsmAp( *this ); + + /* Concatentate duplicates onto the end up until before the last. */ + for ( int i = 1; i < times-1; i++ ) { + FsmAp *dup = new FsmAp( *copyFrom ); + doConcat( dup, 0, false ); + } + + /* Now use the copyFrom on the end. */ + doConcat( copyFrom, 0, false ); +} + +void FsmAp::optionalRepeatOp( int times ) +{ + /* Must be 1 and up. 0 produces null machine and requires deleting this. */ + assert( times > 0 ); + + /* A repeat of one optional merely allows zero string. */ + if ( times == 1 ) { + setFinState( startState ); + return; + } + + /* Make a machine to make copies from. */ + FsmAp *copyFrom = new FsmAp( *this ); + + /* The state set used in the from end of the concatentation. Starts with + * the initial final state set, then after each concatenation, gets set to + * the the final states that come from the the duplicate. */ + StateSet lastFinSet( finStateSet ); + + /* Set the initial state to zero to allow zero copies. */ + setFinState( startState ); + + /* Concatentate duplicates onto the end up until before the last. */ + for ( int i = 1; i < times-1; i++ ) { + /* Make a duplicate for concating and set the fin bits to graph 2 so we + * can pick out it's final states after the optional style concat. */ + FsmAp *dup = new FsmAp( *copyFrom ); + dup->setFinBits( STB_GRAPH2 ); + doConcat( dup, &lastFinSet, true ); + + /* Clear the last final state set and make the new one by taking only + * the final states that come from graph 2.*/ + lastFinSet.empty(); + for ( int i = 0; i < finStateSet.length(); i++ ) { + /* If the state came from graph 2, add it to the last set and clear + * the bits. */ + StateAp *fs = finStateSet[i]; + if ( fs->stateBits & STB_GRAPH2 ) { + lastFinSet.insert( fs ); + fs->stateBits &= ~STB_GRAPH2; + } + } + } + + /* Now use the copyFrom on the end, no bits set, no bits to clear. */ + doConcat( copyFrom, &lastFinSet, true ); +} + + +/* Fsm concatentation worker. Supports treating the concatentation as optional, + * which essentially leaves the final states of machine one as final. */ +void FsmAp::doConcat( FsmAp *other, StateSet *fromStates, bool optional ) +{ + /* For the merging process. */ + StateSet finStateSetCopy, startStateSet; + MergeData md; + + /* Turn on misfit accounting for both graphs. */ + setMisfitAccounting( true ); + other->setMisfitAccounting( true ); + + /* Get the other's start state. */ + StateAp *otherStartState = other->startState; + + /* Unset other's start state before bringing in the entry points. */ + other->unsetStartState(); + + /* Bring in the rest of other's entry points. */ + copyInEntryPoints( other ); + other->entryPoints.empty(); + + /* Bring in other's states into our state lists. */ + stateList.append( other->stateList ); + misfitList.append( other->misfitList ); + + /* If from states is not set, then get a copy of our final state set before + * we clobber it and use it instead. */ + if ( fromStates == 0 ) { + finStateSetCopy = finStateSet; + fromStates = &finStateSetCopy; + } + + /* Unset all of our final states and get the final states from other. */ + if ( !optional ) + unsetAllFinStates(); + finStateSet.insert( other->finStateSet ); + + /* Since other's lists are empty, we can delete the fsm without + * affecting any states. */ + delete other; + + /* Merge our former final states with the start state of other. */ + for ( int i = 0; i < fromStates->length(); i++ ) { + StateAp *state = fromStates->data[i]; + + /* Merge the former final state with other's start state. */ + mergeStatesLeaving( md, state, otherStartState ); + + /* If the former final state was not reset final then we must clear + * the state's out trans data. If it got reset final then it gets to + * keep its out trans data. This must be done before fillInStates gets + * called to prevent the data from being sourced. */ + if ( ! state->isFinState() ) + clearOutData( state ); + } + + /* Fill in any new states made from merging. */ + fillInStates( md ); + + /* Remove the misfits and turn off misfit accounting. */ + removeMisfits(); + setMisfitAccounting( false ); +} + +/* Concatenates other to the end of this machine. Other is deleted. Any + * transitions made leaving this machine and entering into other are notified + * that they are leaving transitions by having the leavingFromState callback + * invoked. */ +void FsmAp::concatOp( FsmAp *other ) +{ + /* Assert same signedness and return graph concatenation op. */ + doConcat( other, 0, false ); +} + + +void FsmAp::doOr( FsmAp *other ) +{ + /* For the merging process. */ + MergeData md; + + /* Build a state set consisting of both start states */ + StateSet startStateSet; + startStateSet.insert( startState ); + startStateSet.insert( other->startState ); + + /* Both of the original start states loose their start state status. */ + unsetStartState(); + other->unsetStartState(); + + /* Bring in the rest of other's entry points. */ + copyInEntryPoints( other ); + other->entryPoints.empty(); + + /* Merge the lists. This will move all the states from other + * into this. No states will be deleted. */ + stateList.append( other->stateList ); + misfitList.append( other->misfitList ); + + /* Move the final set data from other into this. */ + finStateSet.insert(other->finStateSet); + other->finStateSet.empty(); + + /* Since other's list is empty, we can delete the fsm without + * affecting any states. */ + delete other; + + /* Create a new start state. */ + setStartState( addState() ); + + /* Merge the start states. */ + mergeStates( md, startState, startStateSet.data, startStateSet.length() ); + + /* Fill in any new states made from merging. */ + fillInStates( md ); +} + +/* Unions other with this machine. Other is deleted. */ +void FsmAp::unionOp( FsmAp *other ) +{ + /* Turn on misfit accounting for both graphs. */ + setMisfitAccounting( true ); + other->setMisfitAccounting( true ); + + /* Call Worker routine. */ + doOr( other ); + + /* Remove the misfits and turn off misfit accounting. */ + removeMisfits(); + setMisfitAccounting( false ); +} + +/* Intersects other with this machine. Other is deleted. */ +void FsmAp::intersectOp( FsmAp *other ) +{ + /* Turn on misfit accounting for both graphs. */ + setMisfitAccounting( true ); + other->setMisfitAccounting( true ); + + /* Set the fin bits on this and other to want each other. */ + setFinBits( STB_GRAPH1 ); + other->setFinBits( STB_GRAPH2 ); + + /* Call worker Or routine. */ + doOr( other ); + + /* Unset any final states that are no longer to + * be final due to final bits. */ + unsetIncompleteFinals(); + + /* Remove the misfits and turn off misfit accounting. */ + removeMisfits(); + setMisfitAccounting( false ); + + /* Remove states that have no path to a final state. */ + removeDeadEndStates(); +} + +/* Set subtracts other machine from this machine. Other is deleted. */ +void FsmAp::subtractOp( FsmAp *other ) +{ + /* Turn on misfit accounting for both graphs. */ + setMisfitAccounting( true ); + other->setMisfitAccounting( true ); + + /* Set the fin bits of other to be killers. */ + other->setFinBits( STB_GRAPH1 ); + + /* Call worker Or routine. */ + doOr( other ); + + /* Unset any final states that are no longer to + * be final due to final bits. */ + unsetKilledFinals(); + + /* Remove the misfits and turn off misfit accounting. */ + removeMisfits(); + setMisfitAccounting( false ); + + /* Remove states that have no path to a final state. */ + removeDeadEndStates(); +} + +bool FsmAp::inEptVect( EptVect *eptVect, StateAp *state ) +{ + if ( eptVect != 0 ) { + /* Vect is there, walk it looking for state. */ + for ( int i = 0; i < eptVect->length(); i++ ) { + if ( eptVect->data[i].targ == state ) + return true; + } + } + return false; +} + +/* Fill epsilon vectors in a root state from a given starting point. Epmploys + * a depth first search through the graph of epsilon transitions. */ +void FsmAp::epsilonFillEptVectFrom( StateAp *root, StateAp *from, bool parentLeaving ) +{ + /* Walk the epsilon transitions out of the state. */ + for ( EpsilonTrans::Iter ep = from->epsilonTrans; ep.lte(); ep++ ) { + /* Find the entry point, if the it does not resove, ignore it. */ + EntryMapEl *enLow, *enHigh; + if ( entryPoints.findMulti( *ep, enLow, enHigh ) ) { + /* Loop the targets. */ + for ( EntryMapEl *en = enLow; en <= enHigh; en++ ) { + /* Do not add the root or states already in eptVect. */ + StateAp *targ = en->value; + if ( targ != from && !inEptVect(root->eptVect, targ) ) { + /* Maybe need to create the eptVect. */ + if ( root->eptVect == 0 ) + root->eptVect = new EptVect(); + + /* If moving to a different graph or if any parent is + * leaving then we are leaving. */ + bool leaving = parentLeaving || + root->owningGraph != targ->owningGraph; + + /* All ok, add the target epsilon and recurse. */ + root->eptVect->append( EptVectEl(targ, leaving) ); + epsilonFillEptVectFrom( root, targ, leaving ); + } + } + } + } +} + +void FsmAp::shadowReadWriteStates( MergeData &md ) +{ + /* Init isolatedShadow algorithm data. */ + for ( StateList::Iter st = stateList; st.lte(); st++ ) + st->isolatedShadow = 0; + + /* Any states that may be both read from and written to must + * be shadowed. */ + for ( StateList::Iter st = stateList; st.lte(); st++ ) { + /* Find such states by looping through stateVect lists, which give us + * the states that will be read from. May cause us to visit the states + * that we are interested in more than once. */ + if ( st->eptVect != 0 ) { + /* For all states that will be read from. */ + for ( EptVect::Iter ept = *st->eptVect; ept.lte(); ept++ ) { + /* Check for read and write to the same state. */ + StateAp *targ = ept->targ; + if ( targ->eptVect != 0 ) { + /* State is to be written to, if the shadow is not already + * there, create it. */ + if ( targ->isolatedShadow == 0 ) { + StateAp *shadow = addState(); + mergeStates( md, shadow, targ ); + targ->isolatedShadow = shadow; + } + + /* Write shadow into the state vector so that it is the + * state that the epsilon transition will read from. */ + ept->targ = targ->isolatedShadow; + } + } + } + } +} + +void FsmAp::resolveEpsilonTrans( MergeData &md ) +{ + /* Walk the state list and invoke recursive worker on each state. */ + for ( StateList::Iter st = stateList; st.lte(); st++ ) + epsilonFillEptVectFrom( st, st, false ); + + /* Prevent reading from and writing to of the same state. */ + shadowReadWriteStates( md ); + + /* For all states that have epsilon transitions out, draw the transitions, + * clear the epsilon transitions. */ + for ( StateList::Iter st = stateList; st.lte(); st++ ) { + /* If there is a state vector, then create the pre-merge state. */ + if ( st->eptVect != 0 ) { + /* Merge all the epsilon targets into the state. */ + for ( EptVect::Iter ept = *st->eptVect; ept.lte(); ept++ ) { + if ( ept->leaving ) + mergeStatesLeaving( md, st, ept->targ ); + else + mergeStates( md, st, ept->targ ); + } + + /* Clean up the target list. */ + delete st->eptVect; + st->eptVect = 0; + } + + /* Clear the epsilon transitions vector. */ + st->epsilonTrans.empty(); + } +} + +void FsmAp::epsilonOp() +{ + /* For merging process. */ + MergeData md; + + setMisfitAccounting( true ); + + for ( StateList::Iter st = stateList; st.lte(); st++ ) + st->owningGraph = 0; + + /* Perform merges. */ + resolveEpsilonTrans( md ); + + /* Epsilons can caused merges which leave behind unreachable states. */ + fillInStates( md ); + + /* Remove the misfits and turn off misfit accounting. */ + removeMisfits(); + setMisfitAccounting( false ); +} + +/* Make a new maching by joining together a bunch of machines without making + * any transitions between them. A negative finalId results in there being no + * final id. */ +void FsmAp::joinOp( int startId, int finalId, FsmAp **others, int numOthers ) +{ + /* For the merging process. */ + MergeData md; + + /* Set the owning machines. Start at one. Zero is reserved for the start + * and final states. */ + for ( StateList::Iter st = stateList; st.lte(); st++ ) + st->owningGraph = 1; + for ( int m = 0; m < numOthers; m++ ) { + for ( StateList::Iter st = others[m]->stateList; st.lte(); st++ ) + st->owningGraph = 2+m; + } + + /* All machines loose start state status. */ + unsetStartState(); + for ( int m = 0; m < numOthers; m++ ) + others[m]->unsetStartState(); + + /* Bring the other machines into this. */ + for ( int m = 0; m < numOthers; m++ ) { + /* Bring in the rest of other's entry points. */ + copyInEntryPoints( others[m] ); + others[m]->entryPoints.empty(); + + /* Merge the lists. This will move all the states from other into + * this. No states will be deleted. */ + stateList.append( others[m]->stateList ); + assert( others[m]->misfitList.length() == 0 ); + + /* Move the final set data from other into this. */ + finStateSet.insert( others[m]->finStateSet ); + others[m]->finStateSet.empty(); + + /* Since other's list is empty, we can delete the fsm without + * affecting any states. */ + delete others[m]; + } + + /* Look up the start entry point. */ + EntryMapEl *enLow = 0, *enHigh = 0; + bool findRes = entryPoints.findMulti( startId, enLow, enHigh ); + if ( ! findRes ) { + /* No start state. Set a default one and proceed with the join. Note + * that the result of the join will be a very uninteresting machine. */ + setStartState( addState() ); + } + else { + /* There is at least one start state, create a state that will become + * the new start state. */ + StateAp *newStart = addState(); + setStartState( newStart ); + + /* The start state is in an owning machine class all it's own. */ + newStart->owningGraph = 0; + + /* Create the set of states to merge from. */ + StateSet stateSet; + for ( EntryMapEl *en = enLow; en <= enHigh; en++ ) + stateSet.insert( en->value ); + + /* Merge in the set of start states into the new start state. */ + mergeStates( md, newStart, stateSet.data, stateSet.length() ); + } + + /* Take a copy of the final state set, before unsetting them all. This + * will allow us to call clearOutData on the states that don't get + * final state status back back. */ + StateSet finStateSetCopy = finStateSet; + + /* Now all final states are unset. */ + unsetAllFinStates(); + + if ( finalId >= 0 ) { + /* Create the implicit final state. */ + StateAp *finState = addState(); + setFinState( finState ); + + /* Assign an entry into the final state on the final state entry id. Note + * that there may already be an entry on this id. That's ok. Also set the + * final state owning machine id. It's in a class all it's own. */ + setEntry( finalId, finState ); + finState->owningGraph = 0; + } + + /* Hand over to workers for resolving epsilon trans. This will merge states + * with the targets of their epsilon transitions. */ + resolveEpsilonTrans( md ); + + /* Invoke the relinquish final callback on any states that did not get + * final state status back. */ + for ( StateSet::Iter st = finStateSetCopy; st.lte(); st++ ) { + if ( !((*st)->stateBits & STB_ISFINAL) ) + clearOutData( *st ); + } + + /* Fill in any new states made from merging. */ + fillInStates( md ); + + /* Joining can be messy. Instead of having misfit accounting on (which is + * tricky here) do a full cleaning. */ + removeUnreachableStates(); +} + +void FsmAp::globOp( FsmAp **others, int numOthers ) +{ + /* All other machines loose start states status. */ + for ( int m = 0; m < numOthers; m++ ) + others[m]->unsetStartState(); + + /* Bring the other machines into this. */ + for ( int m = 0; m < numOthers; m++ ) { + /* Bring in the rest of other's entry points. */ + copyInEntryPoints( others[m] ); + others[m]->entryPoints.empty(); + + /* Merge the lists. This will move all the states from other into + * this. No states will be deleted. */ + stateList.append( others[m]->stateList ); + assert( others[m]->misfitList.length() == 0 ); + + /* Move the final set data from other into this. */ + finStateSet.insert( others[m]->finStateSet ); + others[m]->finStateSet.empty(); + + /* Since other's list is empty, we can delete the fsm without + * affecting any states. */ + delete others[m]; + } +} + +void FsmAp::deterministicEntry() +{ + /* For the merging process. */ + MergeData md; + + /* States may loose their entry points, turn on misfit accounting. */ + setMisfitAccounting( true ); + + /* Get a copy of the entry map then clear all the entry points. As we + * iterate the old entry map finding duplicates we will add the entry + * points for the new states that we create. */ + EntryMap prevEntry = entryPoints; + unsetAllEntryPoints(); + + for ( int enId = 0; enId < prevEntry.length(); ) { + /* Count the number of states on this entry key. */ + int highId = enId; + while ( highId < prevEntry.length() && prevEntry[enId].key == prevEntry[highId].key ) + highId += 1; + + int numIds = highId - enId; + if ( numIds == 1 ) { + /* Only a single entry point, just set the entry. */ + setEntry( prevEntry[enId].key, prevEntry[enId].value ); + } + else { + /* Multiple entry points, need to create a new state and merge in + * all the targets of entry points. */ + StateAp *newEntry = addState(); + for ( int en = enId; en < highId; en++ ) + mergeStates( md, newEntry, prevEntry[en].value ); + + /* Add the new state as the single entry point. */ + setEntry( prevEntry[enId].key, newEntry ); + } + + enId += numIds; + } + + /* The old start state may be unreachable. Remove the misfits and turn off + * misfit accounting. */ + removeMisfits(); + setMisfitAccounting( false ); +} + +/* Unset any final states that are no longer to be final due to final bits. */ +void FsmAp::unsetKilledFinals() +{ + /* Duplicate the final state set before we begin modifying it. */ + StateSet fin( finStateSet ); + + for ( int s = 0; s < fin.length(); s++ ) { + /* Check for killing bit. */ + StateAp *state = fin.data[s]; + if ( state->stateBits & STB_GRAPH1 ) { + /* One final state is a killer, set to non-final. */ + unsetFinState( state ); + } + + /* Clear all killing bits. Non final states should never have had those + * state bits set in the first place. */ + state->stateBits &= ~STB_GRAPH1; + } +} + +/* Unset any final states that are no longer to be final due to final bits. */ +void FsmAp::unsetIncompleteFinals() +{ + /* Duplicate the final state set before we begin modifying it. */ + StateSet fin( finStateSet ); + + for ( int s = 0; s < fin.length(); s++ ) { + /* Check for one set but not the other. */ + StateAp *state = fin.data[s]; + if ( state->stateBits & STB_BOTH && + (state->stateBits & STB_BOTH) != STB_BOTH ) + { + /* One state wants the other but it is not there. */ + unsetFinState( state ); + } + + /* Clear wanting bits. Non final states should never have had those + * state bits set in the first place. */ + state->stateBits &= ~STB_BOTH; + } +} + +/* Ensure that the start state is free of entry points (aside from the fact + * that it is the start state). If the start state has entry points then Make a + * new start state by merging with the old one. Useful before modifying start + * transitions. If the existing start state has any entry points other than the + * start state entry then modifying its transitions changes more than the start + * transitions. So isolate the start state by separating it out such that it + * only has start stateness as it's entry point. */ +void FsmAp::isolateStartState( ) +{ + /* For the merging process. */ + MergeData md; + + /* Bail out if the start state is already isolated. */ + if ( isStartStateIsolated() ) + return; + + /* Turn on misfit accounting to possibly catch the old start state. */ + setMisfitAccounting( true ); + + /* This will be the new start state. The existing start + * state is merged with it. */ + StateAp *prevStartState = startState; + unsetStartState(); + setStartState( addState() ); + + /* Merge the new start state with the old one to isolate it. */ + mergeStates( md, startState, prevStartState ); + + /* Stfil and stateDict will be empty because the merging of the old start + * state into the new one will not have any conflicting transitions. */ + assert( md.stateDict.treeSize == 0 ); + assert( md.stfillHead == 0 ); + + /* The old start state may be unreachable. Remove the misfits and turn off + * misfit accounting. */ + removeMisfits(); + setMisfitAccounting( false ); +} + +#ifdef LOG_CONDS +void logCondSpace( CondSpace *condSpace ) +{ + if ( condSpace == 0 ) + cerr << "<empty>"; + else { + for ( CondSet::Iter csi = condSpace->condSet.last(); csi.gtb(); csi-- ) { + if ( ! csi.last() ) + cerr << ','; + (*csi)->actionName( cerr ); + } + } +} + +void logNewExpansion( Expansion *exp ) +{ + cerr << "created expansion:" << endl; + cerr << " range: " << exp->lowKey.getVal() << " .. " << + exp->highKey.getVal() << endl; + + cerr << " fromCondSpace: "; + logCondSpace( exp->fromCondSpace ); + cerr << endl; + cerr << " fromVals: " << exp->fromVals << endl; + + cerr << " toCondSpace: "; + logCondSpace( exp->toCondSpace ); + cerr << endl; + cerr << " toValsList: "; + for ( LongVect::Iter to = exp->toValsList; to.lte(); to++ ) + cerr << " " << *to; + cerr << endl; +} +#endif + + +void FsmAp::findTransExpansions( ExpansionList &expansionList, + StateAp *destState, StateAp *srcState ) +{ + PairIter<TransAp, StateCond> transCond( destState->outList.head, + srcState->stateCondList.head ); + for ( ; !transCond.end(); transCond++ ) { + if ( transCond.userState == RangeOverlap ) { + Expansion *expansion = new Expansion( transCond.s1Tel.lowKey, + transCond.s1Tel.highKey ); + expansion->fromTrans = new TransAp(*transCond.s1Tel.trans); + expansion->fromTrans->fromState = 0; + expansion->fromTrans->toState = transCond.s1Tel.trans->toState; + expansion->fromCondSpace = 0; + expansion->fromVals = 0; + CondSpace *srcCS = transCond.s2Tel.trans->condSpace; + expansion->toCondSpace = srcCS; + + long numTargVals = (1 << srcCS->condSet.length()); + for ( long targVals = 0; targVals < numTargVals; targVals++ ) + expansion->toValsList.append( targVals ); + + #ifdef LOG_CONDS + logNewExpansion( expansion ); + #endif + expansionList.append( expansion ); + } + } +} + +void FsmAp::findCondExpInTrans( ExpansionList &expansionList, StateAp *state, + Key lowKey, Key highKey, CondSpace *fromCondSpace, CondSpace *toCondSpace, + long fromVals, LongVect &toValsList ) +{ + /* Make condition-space low and high keys for searching. */ + TransAp searchTrans; + searchTrans.lowKey = fromCondSpace->baseKey + fromVals * keyOps->alphSize() + + (lowKey - keyOps->minKey); + searchTrans.highKey = fromCondSpace->baseKey + fromVals * keyOps->alphSize() + + (highKey - keyOps->minKey); + searchTrans.prev = searchTrans.next = 0; + + PairIter<TransAp> pairIter( state->outList.head, &searchTrans ); + for ( ; !pairIter.end(); pairIter++ ) { + if ( pairIter.userState == RangeOverlap ) { + /* Need to make character-space low and high keys from the range + * overlap for the expansion object. */ + Key expLowKey = pairIter.s1Tel.lowKey - fromCondSpace->baseKey - fromVals * + keyOps->alphSize() + keyOps->minKey; + Key expHighKey = pairIter.s1Tel.highKey - fromCondSpace->baseKey - fromVals * + keyOps->alphSize() + keyOps->minKey; + + Expansion *expansion = new Expansion( expLowKey, expHighKey ); + expansion->fromTrans = new TransAp(*pairIter.s1Tel.trans); + expansion->fromTrans->fromState = 0; + expansion->fromTrans->toState = pairIter.s1Tel.trans->toState; + expansion->fromCondSpace = fromCondSpace; + expansion->fromVals = fromVals; + expansion->toCondSpace = toCondSpace; + expansion->toValsList = toValsList; + + expansionList.append( expansion ); + #ifdef LOG_CONDS + logNewExpansion( expansion ); + #endif + } + } +} + +void FsmAp::findCondExpansions( ExpansionList &expansionList, + StateAp *destState, StateAp *srcState ) +{ + PairIter<StateCond, StateCond> condCond( destState->stateCondList.head, + srcState->stateCondList.head ); + for ( ; !condCond.end(); condCond++ ) { + if ( condCond.userState == RangeOverlap ) { + /* Loop over all existing condVals . */ + CondSet &destCS = condCond.s1Tel.trans->condSpace->condSet; + long destLen = destCS.length(); + + /* Find the items in src cond set that are not in dest + * cond set. These are the items that we must expand. */ + CondSet srcOnlyCS = condCond.s2Tel.trans->condSpace->condSet; + for ( CondSet::Iter dcsi = destCS; dcsi.lte(); dcsi++ ) + srcOnlyCS.remove( *dcsi ); + long srcOnlyLen = srcOnlyCS.length(); + + if ( srcOnlyCS.length() > 0 ) { + #ifdef LOG_CONDS + cerr << "there are " << srcOnlyCS.length() << " item(s) that are " + "only in the srcCS" << endl; + #endif + + CondSet mergedCS = destCS; + mergedCS.insert( condCond.s2Tel.trans->condSpace->condSet ); + + CondSpace *fromCondSpace = addCondSpace( destCS ); + CondSpace *toCondSpace = addCondSpace( mergedCS ); + + /* Loop all values in the dest space. */ + for ( long destVals = 0; destVals < (1 << destLen); destVals++ ) { + long basicVals = 0; + for ( CondSet::Iter csi = destCS; csi.lte(); csi++ ) { + if ( destVals & (1 << csi.pos()) ) { + Action **cim = mergedCS.find( *csi ); + long bitPos = (cim - mergedCS.data); + basicVals |= 1 << bitPos; + } + } + + /* Loop all new values. */ + LongVect expandToVals; + for ( long soVals = 0; soVals < (1 << srcOnlyLen); soVals++ ) { + long targVals = basicVals; + for ( CondSet::Iter csi = srcOnlyCS; csi.lte(); csi++ ) { + if ( soVals & (1 << csi.pos()) ) { + Action **cim = mergedCS.find( *csi ); + long bitPos = (cim - mergedCS.data); + targVals |= 1 << bitPos; + } + } + expandToVals.append( targVals ); + } + + findCondExpInTrans( expansionList, destState, + condCond.s1Tel.lowKey, condCond.s1Tel.highKey, + fromCondSpace, toCondSpace, destVals, expandToVals ); + } + } + } + } +} + +void FsmAp::doExpand( MergeData &md, StateAp *destState, ExpansionList &expList1 ) +{ + for ( ExpansionList::Iter exp = expList1; exp.lte(); exp++ ) { + for ( LongVect::Iter to = exp->toValsList; to.lte(); to++ ) { + long targVals = *to; + + /* We will use the copy of the transition that was made when the + * expansion was created. It will get used multiple times. Each + * time we must set up the keys, everything else is constant and + * and already prepared. */ + TransAp *srcTrans = exp->fromTrans; + + srcTrans->lowKey = exp->toCondSpace->baseKey + + targVals * keyOps->alphSize() + (exp->lowKey - keyOps->minKey); + srcTrans->highKey = exp->toCondSpace->baseKey + + targVals * keyOps->alphSize() + (exp->highKey - keyOps->minKey); + + TransList srcList; + srcList.append( srcTrans ); + outTransCopy( md, destState, srcList.head ); + srcList.abandon(); + } + } +} + + +void FsmAp::doRemove( MergeData &md, StateAp *destState, ExpansionList &expList1 ) +{ + for ( ExpansionList::Iter exp = expList1; exp.lte(); exp++ ) { + Removal removal; + if ( exp->fromCondSpace == 0 ) { + removal.lowKey = exp->lowKey; + removal.highKey = exp->highKey; + } + else { + removal.lowKey = exp->fromCondSpace->baseKey + + exp->fromVals * keyOps->alphSize() + (exp->lowKey - keyOps->minKey); + removal.highKey = exp->fromCondSpace->baseKey + + exp->fromVals * keyOps->alphSize() + (exp->highKey - keyOps->minKey); + } + removal.next = 0; + + TransList destList; + PairIter<TransAp, Removal> pairIter( destState->outList.head, &removal ); + for ( ; !pairIter.end(); pairIter++ ) { + switch ( pairIter.userState ) { + case RangeInS1: { + TransAp *destTrans = pairIter.s1Tel.trans; + destTrans->lowKey = pairIter.s1Tel.lowKey; + destTrans->highKey = pairIter.s1Tel.highKey; + destList.append( destTrans ); + break; + } + case RangeInS2: + break; + case RangeOverlap: { + TransAp *trans = pairIter.s1Tel.trans; + detachTrans( trans->fromState, trans->toState, trans ); + delete trans; + break; + } + case BreakS1: { + pairIter.s1Tel.trans = dupTrans( destState, + pairIter.s1Tel.trans ); + break; + } + case BreakS2: + break; + } + } + destState->outList.transfer( destList ); + } +} + +void FsmAp::mergeStateConds( StateAp *destState, StateAp *srcState ) +{ + StateCondList destList; + PairIter<StateCond> pairIter( destState->stateCondList.head, + srcState->stateCondList.head ); + for ( ; !pairIter.end(); pairIter++ ) { + switch ( pairIter.userState ) { + case RangeInS1: { + StateCond *destCond = pairIter.s1Tel.trans; + destCond->lowKey = pairIter.s1Tel.lowKey; + destCond->highKey = pairIter.s1Tel.highKey; + destList.append( destCond ); + break; + } + case RangeInS2: { + StateCond *newCond = new StateCond( *pairIter.s2Tel.trans ); + newCond->lowKey = pairIter.s2Tel.lowKey; + newCond->highKey = pairIter.s2Tel.highKey; + destList.append( newCond ); + break; + } + case RangeOverlap: { + StateCond *destCond = pairIter.s1Tel.trans; + StateCond *srcCond = pairIter.s2Tel.trans; + CondSet mergedCondSet; + mergedCondSet.insert( destCond->condSpace->condSet ); + mergedCondSet.insert( srcCond->condSpace->condSet ); + destCond->condSpace = addCondSpace( mergedCondSet ); + + destCond->lowKey = pairIter.s1Tel.lowKey; + destCond->highKey = pairIter.s1Tel.highKey; + destList.append( destCond ); + break; + } + case BreakS1: + pairIter.s1Tel.trans = new StateCond( *pairIter.s1Tel.trans ); + break; + + case BreakS2: + break; + } + } + destState->stateCondList.transfer( destList ); +} + +/* A state merge which represents the drawing in of leaving transitions. If + * there is any out data then we duplicate the source state, transfer the out + * data, then merge in the state. The new state will be reaped because it will + * not be given any in transitions. */ +void FsmAp::mergeStatesLeaving( MergeData &md, StateAp *destState, StateAp *srcState ) +{ + if ( !hasOutData( destState ) ) + mergeStates( md, destState, srcState ); + else { + StateAp *ssMutable = addState(); + mergeStates( md, ssMutable, srcState ); + transferOutData( ssMutable, destState ); + + for ( OutCondSet::Iter cond = destState->outCondSet; cond.lte(); cond++ ) + embedCondition( md, ssMutable, cond->action, cond->sense ); + + mergeStates( md, destState, ssMutable ); + } +} + +void FsmAp::mergeStates( MergeData &md, StateAp *destState, + StateAp **srcStates, int numSrc ) +{ + for ( int s = 0; s < numSrc; s++ ) + mergeStates( md, destState, srcStates[s] ); +} + +void FsmAp::mergeStates( MergeData &md, StateAp *destState, StateAp *srcState ) +{ + ExpansionList expList1; + ExpansionList expList2; + + findTransExpansions( expList1, destState, srcState ); + findCondExpansions( expList1, destState, srcState ); + findTransExpansions( expList2, srcState, destState ); + findCondExpansions( expList2, srcState, destState ); + + mergeStateConds( destState, srcState ); + + outTransCopy( md, destState, srcState->outList.head ); + + doExpand( md, destState, expList1 ); + doExpand( md, destState, expList2 ); + + doRemove( md, destState, expList1 ); + doRemove( md, destState, expList2 ); + + expList1.empty(); + expList2.empty(); + + /* Get its bits and final state status. */ + destState->stateBits |= ( srcState->stateBits & ~STB_ISFINAL ); + if ( srcState->isFinState() ) + setFinState( destState ); + + /* Draw in any properties of srcState into destState. */ + if ( srcState == destState ) { + /* Duplicate the list to protect against write to source. The + * priorities sets are not copied in because that would have no + * effect. */ + destState->epsilonTrans.append( EpsilonTrans( srcState->epsilonTrans ) ); + + /* Get all actions, duplicating to protect against write to source. */ + destState->toStateActionTable.setActions( + ActionTable( srcState->toStateActionTable ) ); + destState->fromStateActionTable.setActions( + ActionTable( srcState->fromStateActionTable ) ); + destState->outActionTable.setActions( ActionTable( srcState->outActionTable ) ); + destState->outCondSet.insert( OutCondSet( srcState->outCondSet ) ); + destState->errActionTable.setActions( ErrActionTable( srcState->errActionTable ) ); + destState->eofActionTable.setActions( ActionTable( srcState->eofActionTable ) ); + } + else { + /* Get the epsilons, out priorities. */ + destState->epsilonTrans.append( srcState->epsilonTrans ); + destState->outPriorTable.setPriors( srcState->outPriorTable ); + + /* Get all actions. */ + destState->toStateActionTable.setActions( srcState->toStateActionTable ); + destState->fromStateActionTable.setActions( srcState->fromStateActionTable ); + destState->outActionTable.setActions( srcState->outActionTable ); + destState->outCondSet.insert( srcState->outCondSet ); + destState->errActionTable.setActions( srcState->errActionTable ); + destState->eofActionTable.setActions( srcState->eofActionTable ); + } +} + +void FsmAp::fillInStates( MergeData &md ) +{ + /* Merge any states that are awaiting merging. This will likey cause + * other states to be added to the stfil list. */ + StateAp *state = md.stfillHead; + while ( state != 0 ) { + StateSet *stateSet = &state->stateDictEl->stateSet; + mergeStates( md, state, stateSet->data, stateSet->length() ); + state = state->alg.next; + } + + /* Delete the state sets of all states that are on the fill list. */ + state = md.stfillHead; + while ( state != 0 ) { + /* Delete and reset the state set. */ + delete state->stateDictEl; + state->stateDictEl = 0; + + /* Next state in the stfill list. */ + state = state->alg.next; + } + + /* StateDict will still have its ptrs/size set but all of it's element + * will be deleted so we don't need to clean it up. */ +} + +void FsmAp::findEmbedExpansions( ExpansionList &expansionList, + StateAp *destState, Action *condAction, bool sense ) +{ + StateCondList destList; + PairIter<TransAp, StateCond> transCond( destState->outList.head, + destState->stateCondList.head ); + for ( ; !transCond.end(); transCond++ ) { + switch ( transCond.userState ) { + case RangeInS1: { + if ( transCond.s1Tel.lowKey <= keyOps->maxKey ) { + assert( transCond.s1Tel.highKey <= keyOps->maxKey ); + + /* Make a new state cond. */ + StateCond *newStateCond = new StateCond( transCond.s1Tel.lowKey, + transCond.s1Tel.highKey ); + newStateCond->condSpace = addCondSpace( CondSet( condAction ) ); + destList.append( newStateCond ); + + /* Create the expansion. */ + Expansion *expansion = new Expansion( transCond.s1Tel.lowKey, + transCond.s1Tel.highKey ); + expansion->fromTrans = new TransAp(*transCond.s1Tel.trans); + expansion->fromTrans->fromState = 0; + expansion->fromTrans->toState = transCond.s1Tel.trans->toState; + expansion->fromCondSpace = 0; + expansion->fromVals = 0; + expansion->toCondSpace = newStateCond->condSpace; + expansion->toValsList.append( sense?1:0 ); + #ifdef LOG_CONDS + logNewExpansion( expansion ); + #endif + expansionList.append( expansion ); + } + break; + } + case RangeInS2: { + /* Enhance state cond and find the expansion. */ + StateCond *stateCond = transCond.s2Tel.trans; + stateCond->lowKey = transCond.s2Tel.lowKey; + stateCond->highKey = transCond.s2Tel.highKey; + + CondSet &destCS = stateCond->condSpace->condSet; + long destLen = destCS.length(); + CondSpace *fromCondSpace = stateCond->condSpace; + + CondSet mergedCS = destCS; + mergedCS.insert( condAction ); + CondSpace *toCondSpace = addCondSpace( mergedCS ); + stateCond->condSpace = toCondSpace; + destList.append( stateCond ); + + /* Loop all values in the dest space. */ + for ( long destVals = 0; destVals < (1 << destLen); destVals++ ) { + long basicVals = 0; + for ( CondSet::Iter csi = destCS; csi.lte(); csi++ ) { + if ( destVals & (1 << csi.pos()) ) { + Action **cim = mergedCS.find( *csi ); + long bitPos = (cim - mergedCS.data); + basicVals |= 1 << bitPos; + } + } + + long targVals = basicVals; + Action **cim = mergedCS.find( condAction ); + long bitPos = (cim - mergedCS.data); + targVals |= (sense?1:0) << bitPos; + + LongVect expandToVals( targVals ); + findCondExpInTrans( expansionList, destState, + transCond.s2Tel.lowKey, transCond.s2Tel.highKey, + fromCondSpace, toCondSpace, destVals, expandToVals ); + } + break; + } + + + case RangeOverlap: + case BreakS1: + case BreakS2: + assert( false ); + break; + } + } + + destState->stateCondList.transfer( destList ); +} + +void FsmAp::embedCondition( StateAp *state, Action *condAction, bool sense ) +{ + MergeData md; + ExpansionList expList; + + /* Turn on misfit accounting to possibly catch the old start state. */ + setMisfitAccounting( true ); + + /* Worker. */ + embedCondition( md, state, condAction, sense ); + + /* Fill in any states that were newed up as combinations of others. */ + fillInStates( md ); + + /* Remove the misfits and turn off misfit accounting. */ + removeMisfits(); + setMisfitAccounting( false ); +} + +void FsmAp::embedCondition( MergeData &md, StateAp *state, Action *condAction, bool sense ) +{ + ExpansionList expList; + + findEmbedExpansions( expList, state, condAction, sense ); + doExpand( md, state, expList ); + doRemove( md, state, expList ); + expList.empty(); +} + +/* Check if a machine defines a single character. This is useful in validating + * ranges and machines to export. */ +bool FsmAp::checkSingleCharMachine() +{ + /* Must have two states. */ + if ( stateList.length() != 2 ) + return false; + /* The start state cannot be final. */ + if ( startState->isFinState() ) + return false; + /* There should be only one final state. */ + if ( finStateSet.length() != 1 ) + return false; + /* The final state cannot have any transitions out. */ + if ( finStateSet[0]->outList.length() != 0 ) + return false; + /* The start state should have only one transition out. */ + if ( startState->outList.length() != 1 ) + return false; + /* The singe transition out of the start state should not be a range. */ + TransAp *startTrans = startState->outList.head; + if ( startTrans->lowKey != startTrans->highKey ) + return false; + return true; +} + diff --git a/contrib/tools/ragel6/fsmgraph.h b/contrib/tools/ragel6/fsmgraph.h new file mode 100644 index 0000000000..c7e676b309 --- /dev/null +++ b/contrib/tools/ragel6/fsmgraph.h @@ -0,0 +1,1527 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _FSMGRAPH_H +#define _FSMGRAPH_H + +#include "config.h" +#include <assert.h> +#include <iostream> +#include <string> +#include "common.h" +#include "vector.h" +#include "bstset.h" +#include "compare.h" +#include "avltree.h" +#include "dlist.h" +#include "bstmap.h" +#include "sbstmap.h" +#include "sbstset.h" +#include "sbsttable.h" +#include "avlset.h" +#include "avlmap.h" +#include "ragel.h" + +//#define LOG_CONDS + +/* Flags that control merging. */ +#define STB_GRAPH1 0x01 +#define STB_GRAPH2 0x02 +#define STB_BOTH 0x03 +#define STB_ISFINAL 0x04 +#define STB_ISMARKED 0x08 +#define STB_ONLIST 0x10 + +using std::ostream; + +struct TransAp; +struct StateAp; +struct FsmAp; +struct Action; +struct LongestMatchPart; +struct LengthDef; + +/* State list element for unambiguous access to list element. */ +struct FsmListEl +{ + StateAp *prev, *next; +}; + +/* This is the marked index for a state pair. Used in minimization. It keeps + * track of whether or not the state pair is marked. */ +struct MarkIndex +{ + MarkIndex(int states); + ~MarkIndex(); + + void markPair(int state1, int state2); + bool isPairMarked(int state1, int state2); + +private: + int numStates; + bool *array; +}; + +extern KeyOps *keyOps; + +/* Transistion Action Element. */ +typedef SBstMapEl< int, Action* > ActionTableEl; + +/* Nodes in the tree that use this action. */ +struct NameInst; +struct InlineList; +typedef Vector<NameInst*> ActionRefs; + +/* Element in list of actions. Contains the string for the code to exectute. */ +struct Action +: + public DListEl<Action>, + public AvlTreeEl<Action> +{ +public: + + Action( const InputLoc &loc, const char *name, InlineList *inlineList, int condId ) + : + loc(loc), + name(name), + inlineList(inlineList), + actionId(-1), + numTransRefs(0), + numToStateRefs(0), + numFromStateRefs(0), + numEofRefs(0), + numCondRefs(0), + anyCall(false), + isLmAction(false), + condId(condId) + { + } + + /* Key for action dictionary. */ + const char *getKey() const { return name; } + + /* Data collected during parse. */ + InputLoc loc; + const char *name; + InlineList *inlineList; + int actionId; + + void actionName( ostream &out ) + { + if ( name != 0 ) + out << name; + else + out << loc.line << ":" << loc.col; + } + + /* Places in the input text that reference the action. */ + ActionRefs actionRefs; + + /* Number of references in the final machine. */ + int numRefs() + { return numTransRefs + numToStateRefs + numFromStateRefs + numEofRefs; } + int numTransRefs; + int numToStateRefs; + int numFromStateRefs; + int numEofRefs; + int numCondRefs; + bool anyCall; + + bool isLmAction; + int condId; +}; + +struct CmpCondId +{ + static inline int compare( const Action *cond1, const Action *cond2 ) + { + if ( cond1->condId < cond2->condId ) + return -1; + else if ( cond1->condId > cond2->condId ) + return 1; + return 0; + } +}; + +/* A list of actions. */ +typedef DList<Action> ActionList; +typedef AvlTree<Action, char *, CmpStr> ActionDict; + +/* Structure for reverse action mapping. */ +struct RevActionMapEl +{ + char *name; + InputLoc location; +}; + + +/* Transition Action Table. */ +struct ActionTable + : public SBstMap< int, Action*, CmpOrd<int> > +{ + void setAction( int ordering, Action *action ); + void setActions( int *orderings, Action **actions, int nActs ); + void setActions( const ActionTable &other ); + + bool hasAction( Action *action ); +}; + +typedef SBstSet< Action*, CmpOrd<Action*> > ActionSet; +typedef CmpSTable< Action*, CmpOrd<Action*> > CmpActionSet; + +/* Transistion Action Element. */ +typedef SBstMapEl< int, LongestMatchPart* > LmActionTableEl; + +/* Transition Action Table. */ +struct LmActionTable + : public SBstMap< int, LongestMatchPart*, CmpOrd<int> > +{ + void setAction( int ordering, LongestMatchPart *action ); + void setActions( const LmActionTable &other ); +}; + +/* Compare of a whole action table element (key & value). */ +struct CmpActionTableEl +{ + static int compare( const ActionTableEl &action1, + const ActionTableEl &action2 ) + { + if ( action1.key < action2.key ) + return -1; + else if ( action1.key > action2.key ) + return 1; + else if ( action1.value < action2.value ) + return -1; + else if ( action1.value > action2.value ) + return 1; + return 0; + } +}; + +/* Compare for ActionTable. */ +typedef CmpSTable< ActionTableEl, CmpActionTableEl > CmpActionTable; + +/* Compare of a whole lm action table element (key & value). */ +struct CmpLmActionTableEl +{ + static int compare( const LmActionTableEl &lmAction1, + const LmActionTableEl &lmAction2 ) + { + if ( lmAction1.key < lmAction2.key ) + return -1; + else if ( lmAction1.key > lmAction2.key ) + return 1; + else if ( lmAction1.value < lmAction2.value ) + return -1; + else if ( lmAction1.value > lmAction2.value ) + return 1; + return 0; + } +}; + +/* Compare for ActionTable. */ +typedef CmpSTable< LmActionTableEl, CmpLmActionTableEl > CmpLmActionTable; + +/* Action table element for error action tables. Adds the encoding of transfer + * point. */ +struct ErrActionTableEl +{ + ErrActionTableEl( Action *action, int ordering, int transferPoint ) + : ordering(ordering), action(action), transferPoint(transferPoint) { } + + /* Ordering and id of the action embedding. */ + int ordering; + Action *action; + + /* Id of point of transfere from Error action table to transtions and + * eofActionTable. */ + int transferPoint; + + int getKey() const { return ordering; } +}; + +struct ErrActionTable + : public SBstTable< ErrActionTableEl, int, CmpOrd<int> > +{ + void setAction( int ordering, Action *action, int transferPoint ); + void setActions( const ErrActionTable &other ); +}; + +/* Compare of an error action table element (key & value). */ +struct CmpErrActionTableEl +{ + static int compare( const ErrActionTableEl &action1, + const ErrActionTableEl &action2 ) + { + if ( action1.ordering < action2.ordering ) + return -1; + else if ( action1.ordering > action2.ordering ) + return 1; + else if ( action1.action < action2.action ) + return -1; + else if ( action1.action > action2.action ) + return 1; + else if ( action1.transferPoint < action2.transferPoint ) + return -1; + else if ( action1.transferPoint > action2.transferPoint ) + return 1; + return 0; + } +}; + +/* Compare for ErrActionTable. */ +typedef CmpSTable< ErrActionTableEl, CmpErrActionTableEl > CmpErrActionTable; + + +/* Descibe a priority, shared among PriorEls. + * Has key and whether or not used. */ +struct PriorDesc +{ + int key; + int priority; +}; + +/* Element in the arrays of priorities for transitions and arrays. Ordering is + * unique among instantiations of machines, desc is shared. */ +struct PriorEl +{ + PriorEl( int ordering, PriorDesc *desc ) + : ordering(ordering), desc(desc) { } + + int ordering; + PriorDesc *desc; +}; + +/* Compare priority elements, which are ordered by the priority descriptor + * key. */ +struct PriorElCmp +{ + static inline int compare( const PriorEl &pel1, const PriorEl &pel2 ) + { + if ( pel1.desc->key < pel2.desc->key ) + return -1; + else if ( pel1.desc->key > pel2.desc->key ) + return 1; + else + return 0; + } +}; + + +/* Priority Table. */ +struct PriorTable + : public SBstSet< PriorEl, PriorElCmp > +{ + void setPrior( int ordering, PriorDesc *desc ); + void setPriors( const PriorTable &other ); +}; + +/* Compare of prior table elements for distinguising state data. */ +struct CmpPriorEl +{ + static inline int compare( const PriorEl &pel1, const PriorEl &pel2 ) + { + if ( pel1.desc < pel2.desc ) + return -1; + else if ( pel1.desc > pel2.desc ) + return 1; + else if ( pel1.ordering < pel2.ordering ) + return -1; + else if ( pel1.ordering > pel2.ordering ) + return 1; + return 0; + } +}; + +/* Compare of PriorTable distinguising state data. Using a compare of the + * pointers is a little more strict than it needs be. It requires that + * prioritiy tables have the exact same set of priority assignment operators + * (from the input lang) to be considered equal. + * + * Really only key-value pairs need be tested and ordering be merged. However + * this would require that in the fuseing of states, priority descriptors be + * chosen for the new fused state based on priority. Since the out transition + * lists and ranges aren't necessarily going to line up, this is more work for + * little gain. Final compression resets all priorities first, so this would + * only be useful for compression at every operator, which is only an + * undocumented test feature. + */ +typedef CmpSTable<PriorEl, CmpPriorEl> CmpPriorTable; + +/* Plain action list that imposes no ordering. */ +typedef Vector<int> TransFuncList; + +/* Comparison for TransFuncList. */ +typedef CmpTable< int, CmpOrd<int> > TransFuncListCompare; + +/* Transition class that implements actions and priorities. */ +struct TransAp +{ + TransAp() : fromState(0), toState(0) {} + TransAp( const TransAp &other ) : + lowKey(other.lowKey), + highKey(other.highKey), + fromState(0), toState(0), + actionTable(other.actionTable), + priorTable(other.priorTable), + lmActionTable(other.lmActionTable) {} + + Key lowKey, highKey; + StateAp *fromState; + StateAp *toState; + + /* Pointers for outlist. */ + TransAp *prev, *next; + + /* Pointers for in-list. */ + TransAp *ilprev, *ilnext; + + /* The function table and priority for the transition. */ + ActionTable actionTable; + PriorTable priorTable; + + LmActionTable lmActionTable; +}; + +/* In transition list. Like DList except only has head pointers, which is all + * that is required. Insertion and deletion is handled by the graph. This + * class provides the iterator of a single list. */ +struct TransInList +{ + TransInList() : head(0) { } + + TransAp *head; + + struct Iter + { + /* Default construct. */ + Iter() : ptr(0) { } + + /* Construct, assign from a list. */ + Iter( const TransInList &il ) : ptr(il.head) { } + Iter &operator=( const TransInList &dl ) { ptr = dl.head; return *this; } + + /* At the end */ + bool lte() const { return ptr != 0; } + bool end() const { return ptr == 0; } + + /* At the first, last element. */ + bool first() const { return ptr && ptr->ilprev == 0; } + bool last() const { return ptr && ptr->ilnext == 0; } + + /* Cast, dereference, arrow ops. */ + operator TransAp*() const { return ptr; } + TransAp &operator *() const { return *ptr; } + TransAp *operator->() const { return ptr; } + + /* Increment, decrement. */ + inline void operator++(int) { ptr = ptr->ilnext; } + inline void operator--(int) { ptr = ptr->ilprev; } + + /* The iterator is simply a pointer. */ + TransAp *ptr; + }; +}; + +typedef DList<TransAp> TransList; + +/* Set of states, list of states. */ +typedef BstSet<StateAp*> StateSet; +typedef DList<StateAp> StateList; + +/* A element in a state dict. */ +struct StateDictEl +: + public AvlTreeEl<StateDictEl> +{ + StateDictEl(const StateSet &stateSet) + : stateSet(stateSet) { } + + const StateSet &getKey() { return stateSet; } + StateSet stateSet; + StateAp *targState; +}; + +/* Dictionary mapping a set of states to a target state. */ +typedef AvlTree< StateDictEl, StateSet, CmpTable<StateAp*> > StateDict; + +/* Data needed for a merge operation. */ +struct MergeData +{ + MergeData() + : stfillHead(0), stfillTail(0) { } + + StateDict stateDict; + + StateAp *stfillHead; + StateAp *stfillTail; + + void fillListAppend( StateAp *state ); +}; + +struct TransEl +{ + /* Constructors. */ + TransEl() { } + TransEl( Key lowKey, Key highKey ) + : lowKey(lowKey), highKey(highKey) { } + TransEl( Key lowKey, Key highKey, TransAp *value ) + : lowKey(lowKey), highKey(highKey), value(value) { } + + Key lowKey, highKey; + TransAp *value; +}; + +struct CmpKey +{ + static int compare( const Key key1, const Key key2 ) + { + if ( key1 < key2 ) + return -1; + else if ( key1 > key2 ) + return 1; + else + return 0; + } +}; + +/* Vector based set of key items. */ +typedef BstSet<Key, CmpKey> KeySet; + +struct MinPartition +{ + MinPartition() : active(false) { } + + StateList list; + bool active; + + MinPartition *prev, *next; +}; + +/* Epsilon transition stored in a state. Specifies the target */ +typedef Vector<int> EpsilonTrans; + +/* List of states that are to be drawn into this. */ +struct EptVectEl +{ + EptVectEl( StateAp *targ, bool leaving ) + : targ(targ), leaving(leaving) { } + + StateAp *targ; + bool leaving; +}; +typedef Vector<EptVectEl> EptVect; + +/* Set of entry ids that go into this state. */ +typedef BstSet<int> EntryIdSet; + +/* Set of longest match items that may be active in a given state. */ +typedef BstSet<LongestMatchPart*> LmItemSet; + +/* A Conditions which is to be + * transfered on pending out transitions. */ +struct OutCond +{ + OutCond( Action *action, bool sense ) + : action(action), sense(sense) {} + + Action *action; + bool sense; +}; + +struct CmpOutCond +{ + static int compare( const OutCond &outCond1, const OutCond &outCond2 ) + { + if ( outCond1.action < outCond2.action ) + return -1; + else if ( outCond1.action > outCond2.action ) + return 1; + else if ( outCond1.sense < outCond2.sense ) + return -1; + else if ( outCond1.sense > outCond2.sense ) + return 1; + return 0; + } +}; + +/* Set of conditions to be transfered to on pending out transitions. */ +typedef SBstSet< OutCond, CmpOutCond > OutCondSet; +typedef CmpSTable< OutCond, CmpOutCond > CmpOutCondSet; + +/* Conditions. */ +typedef BstSet< Action*, CmpCondId > CondSet; +typedef CmpTable< Action*, CmpCondId > CmpCondSet; + +struct CondSpace + : public AvlTreeEl<CondSpace> +{ + CondSpace( const CondSet &condSet ) + : condSet(condSet) {} + + const CondSet &getKey() { return condSet; } + + CondSet condSet; + Key baseKey; + long condSpaceId; +}; + +typedef Vector<CondSpace*> CondSpaceVect; + +typedef AvlTree<CondSpace, CondSet, CmpCondSet> CondSpaceMap; + +struct StateCond +{ + StateCond( Key lowKey, Key highKey ) : + lowKey(lowKey), highKey(highKey) {} + + Key lowKey; + Key highKey; + CondSpace *condSpace; + + StateCond *prev, *next; +}; + +typedef DList<StateCond> StateCondList; +typedef Vector<long> LongVect; + +struct Expansion +{ + Expansion( Key lowKey, Key highKey ) : + lowKey(lowKey), highKey(highKey), + fromTrans(0), fromCondSpace(0), + toCondSpace(0) {} + + ~Expansion() + { + if ( fromTrans != 0 ) + delete fromTrans; + } + + Key lowKey; + Key highKey; + + TransAp *fromTrans; + CondSpace *fromCondSpace; + long fromVals; + + CondSpace *toCondSpace; + LongVect toValsList; + + Expansion *prev, *next; +}; + +typedef DList<Expansion> ExpansionList; + +struct Removal +{ + Key lowKey; + Key highKey; + + Removal *next; +}; + +struct CondData +{ + CondData() : lastCondKey(0) {} + + /* Condition info. */ + Key lastCondKey; + + CondSpaceMap condSpaceMap; +}; + +extern CondData *condData; + +struct FsmConstructFail +{ + enum Reason + { + CondNoKeySpace + }; + + FsmConstructFail( Reason reason ) + : reason(reason) {} + Reason reason; +}; + +/* State class that implements actions and priorities. */ +struct StateAp +{ + StateAp(); + StateAp(const StateAp &other); + ~StateAp(); + + /* Is the state final? */ + bool isFinState() { return stateBits & STB_ISFINAL; } + + /* Out transition list and the pointer for the default out trans. */ + TransList outList; + + /* In transition Lists. */ + TransInList inList; + + /* Set only during scanner construction when actions are added. NFA to DFA + * code can ignore this. */ + StateAp *eofTarget; + + /* Entry points into the state. */ + EntryIdSet entryIds; + + /* Epsilon transitions. */ + EpsilonTrans epsilonTrans; + + /* Condition info. */ + StateCondList stateCondList; + + /* Number of in transitions from states other than ourselves. */ + int foreignInTrans; + + /* Temporary data for various algorithms. */ + union { + /* When duplicating the fsm we need to map each + * state to the new state representing it. */ + StateAp *stateMap; + + /* When minimizing machines by partitioning, this maps to the group + * the state is in. */ + MinPartition *partition; + + /* When merging states (state machine operations) this next pointer is + * used for the list of states that need to be filled in. */ + StateAp *next; + + /* Identification for printing and stable minimization. */ + int stateNum; + + } alg; + + /* Data used in epsilon operation, maybe fit into alg? */ + StateAp *isolatedShadow; + int owningGraph; + + /* A pointer to a dict element that contains the set of states this state + * represents. This cannot go into alg, because alg.next is used during + * the merging process. */ + StateDictEl *stateDictEl; + + /* When drawing epsilon transitions, holds the list of states to merge + * with. */ + EptVect *eptVect; + + /* Bits controlling the behaviour of the state during collapsing to dfa. */ + int stateBits; + + /* State list elements. */ + StateAp *next, *prev; + + /* + * Priority and Action data. + */ + + /* Out priorities transfered to out transitions. */ + PriorTable outPriorTable; + + /* The following two action tables are distinguished by the fact that when + * toState actions are executed immediatly after transition actions of + * incoming transitions and the current character will be the same as the + * one available then. The fromState actions are executed immediately + * before the transition actions of outgoing transitions and the current + * character is same as the one available then. */ + + /* Actions to execute upon entering into a state. */ + ActionTable toStateActionTable; + + /* Actions to execute when going from the state to the transition. */ + ActionTable fromStateActionTable; + + /* Actions to add to any future transitions that leave via this state. */ + ActionTable outActionTable; + + /* Conditions to add to any future transiions that leave via this sttate. */ + OutCondSet outCondSet; + + /* Error action tables. */ + ErrActionTable errActionTable; + + /* Actions to execute on eof. */ + ActionTable eofActionTable; + + /* Set of longest match items that may be active in this state. */ + LmItemSet lmItemSet; +}; + +template <class ListItem> struct NextTrans +{ + Key lowKey, highKey; + ListItem *trans; + ListItem *next; + + void load() { + if ( trans == 0 ) + next = 0; + else { + next = trans->next; + lowKey = trans->lowKey; + highKey = trans->highKey; + } + } + + void set( ListItem *t ) { + trans = t; + load(); + } + + void increment() { + trans = next; + load(); + } +}; + + +/* Encodes the different states that are meaningful to the of the iterator. */ +enum PairIterUserState +{ + RangeInS1, RangeInS2, + RangeOverlap, + BreakS1, BreakS2 +}; + +template <class ListItem1, class ListItem2 = ListItem1> struct PairIter +{ + /* Encodes the different states that an fsm iterator can be in. */ + enum IterState { + Begin, + ConsumeS1Range, ConsumeS2Range, + OnlyInS1Range, OnlyInS2Range, + S1SticksOut, S1SticksOutBreak, + S2SticksOut, S2SticksOutBreak, + S1DragsBehind, S1DragsBehindBreak, + S2DragsBehind, S2DragsBehindBreak, + ExactOverlap, End + }; + + PairIter( ListItem1 *list1, ListItem2 *list2 ); + + /* Query iterator. */ + bool lte() { return itState != End; } + bool end() { return itState == End; } + void operator++(int) { findNext(); } + void operator++() { findNext(); } + + /* Iterator state. */ + ListItem1 *list1; + ListItem2 *list2; + IterState itState; + PairIterUserState userState; + + NextTrans<ListItem1> s1Tel; + NextTrans<ListItem2> s2Tel; + Key bottomLow, bottomHigh; + ListItem1 *bottomTrans1; + ListItem2 *bottomTrans2; + +private: + void findNext(); +}; + +/* Init the iterator by advancing to the first item. */ +template <class ListItem1, class ListItem2> PairIter<ListItem1, ListItem2>::PairIter( + ListItem1 *list1, ListItem2 *list2 ) +: + list1(list1), + list2(list2), + itState(Begin) +{ + findNext(); +} + +/* Return and re-entry for the co-routine iterators. This should ALWAYS be + * used inside of a block. */ +#define CO_RETURN(label) \ + itState = label; \ + return; \ + entry##label: {} + +/* Return and re-entry for the co-routine iterators. This should ALWAYS be + * used inside of a block. */ +#define CO_RETURN2(label, uState) \ + itState = label; \ + userState = uState; \ + return; \ + entry##label: {} + +/* Advance to the next transition. When returns, trans points to the next + * transition, unless there are no more, in which case end() returns true. */ +template <class ListItem1, class ListItem2> void PairIter<ListItem1, ListItem2>::findNext() +{ + /* Jump into the iterator routine base on the iterator state. */ + switch ( itState ) { + case Begin: goto entryBegin; + case ConsumeS1Range: goto entryConsumeS1Range; + case ConsumeS2Range: goto entryConsumeS2Range; + case OnlyInS1Range: goto entryOnlyInS1Range; + case OnlyInS2Range: goto entryOnlyInS2Range; + case S1SticksOut: goto entryS1SticksOut; + case S1SticksOutBreak: goto entryS1SticksOutBreak; + case S2SticksOut: goto entryS2SticksOut; + case S2SticksOutBreak: goto entryS2SticksOutBreak; + case S1DragsBehind: goto entryS1DragsBehind; + case S1DragsBehindBreak: goto entryS1DragsBehindBreak; + case S2DragsBehind: goto entryS2DragsBehind; + case S2DragsBehindBreak: goto entryS2DragsBehindBreak; + case ExactOverlap: goto entryExactOverlap; + case End: goto entryEnd; + } + +entryBegin: + /* Set up the next structs at the head of the transition lists. */ + s1Tel.set( list1 ); + s2Tel.set( list2 ); + + /* Concurrently scan both out ranges. */ + while ( true ) { + if ( s1Tel.trans == 0 ) { + /* We are at the end of state1's ranges. Process the rest of + * state2's ranges. */ + while ( s2Tel.trans != 0 ) { + /* Range is only in s2. */ + CO_RETURN2( ConsumeS2Range, RangeInS2 ); + s2Tel.increment(); + } + break; + } + else if ( s2Tel.trans == 0 ) { + /* We are at the end of state2's ranges. Process the rest of + * state1's ranges. */ + while ( s1Tel.trans != 0 ) { + /* Range is only in s1. */ + CO_RETURN2( ConsumeS1Range, RangeInS1 ); + s1Tel.increment(); + } + break; + } + /* Both state1's and state2's transition elements are good. + * The signiture of no overlap is a back key being in front of a + * front key. */ + else if ( s1Tel.highKey < s2Tel.lowKey ) { + /* A range exists in state1 that does not overlap with state2. */ + CO_RETURN2( OnlyInS1Range, RangeInS1 ); + s1Tel.increment(); + } + else if ( s2Tel.highKey < s1Tel.lowKey ) { + /* A range exists in state2 that does not overlap with state1. */ + CO_RETURN2( OnlyInS2Range, RangeInS2 ); + s2Tel.increment(); + } + /* There is overlap, must mix the ranges in some way. */ + else if ( s1Tel.lowKey < s2Tel.lowKey ) { + /* Range from state1 sticks out front. Must break it into + * non-overlaping and overlaping segments. */ + bottomLow = s2Tel.lowKey; + bottomHigh = s1Tel.highKey; + s1Tel.highKey = s2Tel.lowKey; + s1Tel.highKey.decrement(); + bottomTrans1 = s1Tel.trans; + + /* Notify the caller that we are breaking s1. This gives them a + * chance to duplicate s1Tel[0,1].value. */ + CO_RETURN2( S1SticksOutBreak, BreakS1 ); + + /* Broken off range is only in s1. */ + CO_RETURN2( S1SticksOut, RangeInS1 ); + + /* Advance over the part sticking out front. */ + s1Tel.lowKey = bottomLow; + s1Tel.highKey = bottomHigh; + s1Tel.trans = bottomTrans1; + } + else if ( s2Tel.lowKey < s1Tel.lowKey ) { + /* Range from state2 sticks out front. Must break it into + * non-overlaping and overlaping segments. */ + bottomLow = s1Tel.lowKey; + bottomHigh = s2Tel.highKey; + s2Tel.highKey = s1Tel.lowKey; + s2Tel.highKey.decrement(); + bottomTrans2 = s2Tel.trans; + + /* Notify the caller that we are breaking s2. This gives them a + * chance to duplicate s2Tel[0,1].value. */ + CO_RETURN2( S2SticksOutBreak, BreakS2 ); + + /* Broken off range is only in s2. */ + CO_RETURN2( S2SticksOut, RangeInS2 ); + + /* Advance over the part sticking out front. */ + s2Tel.lowKey = bottomLow; + s2Tel.highKey = bottomHigh; + s2Tel.trans = bottomTrans2; + } + /* Low ends are even. Are the high ends even? */ + else if ( s1Tel.highKey < s2Tel.highKey ) { + /* Range from state2 goes longer than the range from state1. We + * must break the range from state2 into an evenly overlaping + * segment. */ + bottomLow = s1Tel.highKey; + bottomLow.increment(); + bottomHigh = s2Tel.highKey; + s2Tel.highKey = s1Tel.highKey; + bottomTrans2 = s2Tel.trans; + + /* Notify the caller that we are breaking s2. This gives them a + * chance to duplicate s2Tel[0,1].value. */ + CO_RETURN2( S2DragsBehindBreak, BreakS2 ); + + /* Breaking s2 produces exact overlap. */ + CO_RETURN2( S2DragsBehind, RangeOverlap ); + + /* Advance over the front we just broke off of range 2. */ + s2Tel.lowKey = bottomLow; + s2Tel.highKey = bottomHigh; + s2Tel.trans = bottomTrans2; + + /* Advance over the entire s1Tel. We have consumed it. */ + s1Tel.increment(); + } + else if ( s2Tel.highKey < s1Tel.highKey ) { + /* Range from state1 goes longer than the range from state2. We + * must break the range from state1 into an evenly overlaping + * segment. */ + bottomLow = s2Tel.highKey; + bottomLow.increment(); + bottomHigh = s1Tel.highKey; + s1Tel.highKey = s2Tel.highKey; + bottomTrans1 = s1Tel.trans; + + /* Notify the caller that we are breaking s1. This gives them a + * chance to duplicate s2Tel[0,1].value. */ + CO_RETURN2( S1DragsBehindBreak, BreakS1 ); + + /* Breaking s1 produces exact overlap. */ + CO_RETURN2( S1DragsBehind, RangeOverlap ); + + /* Advance over the front we just broke off of range 1. */ + s1Tel.lowKey = bottomLow; + s1Tel.highKey = bottomHigh; + s1Tel.trans = bottomTrans1; + + /* Advance over the entire s2Tel. We have consumed it. */ + s2Tel.increment(); + } + else { + /* There is an exact overlap. */ + CO_RETURN2( ExactOverlap, RangeOverlap ); + + s1Tel.increment(); + s2Tel.increment(); + } + } + + /* Done, go into end state. */ + CO_RETURN( End ); +} + + +/* Compare lists of epsilon transitions. Entries are name ids of targets. */ +typedef CmpTable< int, CmpOrd<int> > CmpEpsilonTrans; + +/* Compare class for the Approximate minimization. */ +class ApproxCompare +{ +public: + ApproxCompare() { } + int compare( const StateAp *pState1, const StateAp *pState2 ); +}; + +/* Compare class for the initial partitioning of a partition minimization. */ +class InitPartitionCompare +{ +public: + InitPartitionCompare() { } + int compare( const StateAp *pState1, const StateAp *pState2 ); +}; + +/* Compare class for the regular partitioning of a partition minimization. */ +class PartitionCompare +{ +public: + PartitionCompare() { } + int compare( const StateAp *pState1, const StateAp *pState2 ); +}; + +/* Compare class for a minimization that marks pairs. Provides the shouldMark + * routine. */ +class MarkCompare +{ +public: + MarkCompare() { } + bool shouldMark( MarkIndex &markIndex, const StateAp *pState1, + const StateAp *pState2 ); +}; + +/* List of partitions. */ +typedef DList< MinPartition > PartitionList; + +/* List of transtions out of a state. */ +typedef Vector<TransEl> TransListVect; + +/* Entry point map used for keeping track of entry points in a machine. */ +typedef BstSet< int > EntryIdSet; +typedef BstMapEl< int, StateAp* > EntryMapEl; +typedef BstMap< int, StateAp* > EntryMap; +typedef Vector<EntryMapEl> EntryMapBase; + +/* Graph class that implements actions and priorities. */ +struct FsmAp +{ + /* Constructors/Destructors. */ + FsmAp( ); + FsmAp( const FsmAp &graph ); + ~FsmAp(); + + /* The list of states. */ + StateList stateList; + StateList misfitList; + + /* The map of entry points. */ + EntryMap entryPoints; + + /* The start state. */ + StateAp *startState; + + /* Error state, possibly created only when the final machine has been + * created and the XML machine is about to be written. No transitions + * point to this state. */ + StateAp *errState; + + /* The set of final states. */ + StateSet finStateSet; + + /* Misfit Accounting. Are misfits put on a separate list. */ + bool misfitAccounting; + + /* + * Transition actions and priorities. + */ + + /* Set priorities on transtions. */ + void startFsmPrior( int ordering, PriorDesc *prior ); + void allTransPrior( int ordering, PriorDesc *prior ); + void finishFsmPrior( int ordering, PriorDesc *prior ); + void leaveFsmPrior( int ordering, PriorDesc *prior ); + + /* Action setting support. */ + void transferOutActions( StateAp *state ); + void transferErrorActions( StateAp *state, int transferPoint ); + void setErrorActions( StateAp *state, const ActionTable &other ); + void setErrorAction( StateAp *state, int ordering, Action *action ); + + /* Fill all spaces in a transition list with an error transition. */ + void fillGaps( StateAp *state ); + + /* Similar to setErrorAction, instead gives a state to go to on error. */ + void setErrorTarget( StateAp *state, StateAp *target, int *orderings, + Action **actions, int nActs ); + + /* Set actions to execute. */ + void startFsmAction( int ordering, Action *action ); + void allTransAction( int ordering, Action *action ); + void finishFsmAction( int ordering, Action *action ); + void leaveFsmAction( int ordering, Action *action ); + void longMatchAction( int ordering, LongestMatchPart *lmPart ); + + /* Set conditions. */ + CondSpace *addCondSpace( const CondSet &condSet ); + + void findEmbedExpansions( ExpansionList &expansionList, + StateAp *destState, Action *condAction, bool sense ); + void embedCondition( MergeData &md, StateAp *state, Action *condAction, bool sense ); + void embedCondition( StateAp *state, Action *condAction, bool sense ); + + void startFsmCondition( Action *condAction, bool sense ); + void allTransCondition( Action *condAction, bool sense ); + void leaveFsmCondition( Action *condAction, bool sense ); + + /* Set error actions to execute. */ + void startErrorAction( int ordering, Action *action, int transferPoint ); + void allErrorAction( int ordering, Action *action, int transferPoint ); + void finalErrorAction( int ordering, Action *action, int transferPoint ); + void notStartErrorAction( int ordering, Action *action, int transferPoint ); + void notFinalErrorAction( int ordering, Action *action, int transferPoint ); + void middleErrorAction( int ordering, Action *action, int transferPoint ); + + /* Set EOF actions. */ + void startEOFAction( int ordering, Action *action ); + void allEOFAction( int ordering, Action *action ); + void finalEOFAction( int ordering, Action *action ); + void notStartEOFAction( int ordering, Action *action ); + void notFinalEOFAction( int ordering, Action *action ); + void middleEOFAction( int ordering, Action *action ); + + /* Set To State actions. */ + void startToStateAction( int ordering, Action *action ); + void allToStateAction( int ordering, Action *action ); + void finalToStateAction( int ordering, Action *action ); + void notStartToStateAction( int ordering, Action *action ); + void notFinalToStateAction( int ordering, Action *action ); + void middleToStateAction( int ordering, Action *action ); + + /* Set From State actions. */ + void startFromStateAction( int ordering, Action *action ); + void allFromStateAction( int ordering, Action *action ); + void finalFromStateAction( int ordering, Action *action ); + void notStartFromStateAction( int ordering, Action *action ); + void notFinalFromStateAction( int ordering, Action *action ); + void middleFromStateAction( int ordering, Action *action ); + + /* Shift the action ordering of the start transitions to start at + * fromOrder and increase in units of 1. Useful before kleene star + * operation. */ + int shiftStartActionOrder( int fromOrder ); + + /* Clear all priorities from the fsm to so they won't affcet minimization + * of the final fsm. */ + void clearAllPriorities(); + + /* Zero out all the function keys. */ + void nullActionKeys(); + + /* Walk the list of states and verify state properties. */ + void verifyStates(); + + /* Misfit Accounting. Are misfits put on a separate list. */ + void setMisfitAccounting( bool val ) + { misfitAccounting = val; } + + /* Set and Unset a state as final. */ + void setFinState( StateAp *state ); + void unsetFinState( StateAp *state ); + + void setStartState( StateAp *state ); + void unsetStartState( ); + + /* Set and unset a state as an entry point. */ + void setEntry( int id, StateAp *state ); + void changeEntry( int id, StateAp *to, StateAp *from ); + void unsetEntry( int id, StateAp *state ); + void unsetEntry( int id ); + void unsetAllEntryPoints(); + + /* Epsilon transitions. */ + void epsilonTrans( int id ); + void shadowReadWriteStates( MergeData &md ); + + /* + * Basic attaching and detaching. + */ + + /* Common to attaching/detaching list and default. */ + void attachToInList( StateAp *from, StateAp *to, TransAp *&head, TransAp *trans ); + void detachFromInList( StateAp *from, StateAp *to, TransAp *&head, TransAp *trans ); + + /* Attach with a new transition. */ + TransAp *attachNewTrans( StateAp *from, StateAp *to, + Key onChar1, Key onChar2 ); + + /* Attach with an existing transition that already in an out list. */ + void attachTrans( StateAp *from, StateAp *to, TransAp *trans ); + + /* Redirect a transition away from error and towards some state. */ + void redirectErrorTrans( StateAp *from, StateAp *to, TransAp *trans ); + + /* Detach a transition from a target state. */ + void detachTrans( StateAp *from, StateAp *to, TransAp *trans ); + + /* Detach a state from the graph. */ + void detachState( StateAp *state ); + + /* + * NFA to DFA conversion routines. + */ + + /* Duplicate a transition that will dropin to a free spot. */ + TransAp *dupTrans( StateAp *from, TransAp *srcTrans ); + + /* In crossing, two transitions both go to real states. */ + TransAp *fsmAttachStates( MergeData &md, StateAp *from, + TransAp *destTrans, TransAp *srcTrans ); + + /* Two transitions are to be crossed, handle the possibility of either + * going to the error state. */ + TransAp *mergeTrans( MergeData &md, StateAp *from, + TransAp *destTrans, TransAp *srcTrans ); + + /* Compare deterimne relative priorities of two transition tables. */ + int comparePrior( const PriorTable &priorTable1, const PriorTable &priorTable2 ); + + /* Cross a src transition with one that is already occupying a spot. */ + TransAp *crossTransitions( MergeData &md, StateAp *from, + TransAp *destTrans, TransAp *srcTrans ); + + void outTransCopy( MergeData &md, StateAp *dest, TransAp *srcList ); + + void doRemove( MergeData &md, StateAp *destState, ExpansionList &expList1 ); + void doExpand( MergeData &md, StateAp *destState, ExpansionList &expList1 ); + void findCondExpInTrans( ExpansionList &expansionList, StateAp *state, + Key lowKey, Key highKey, CondSpace *fromCondSpace, CondSpace *toCondSpace, + long destVals, LongVect &toValsList ); + void findTransExpansions( ExpansionList &expansionList, + StateAp *destState, StateAp *srcState ); + void findCondExpansions( ExpansionList &expansionList, + StateAp *destState, StateAp *srcState ); + void mergeStateConds( StateAp *destState, StateAp *srcState ); + + /* Merge a set of states into newState. */ + void mergeStates( MergeData &md, StateAp *destState, + StateAp **srcStates, int numSrc ); + void mergeStatesLeaving( MergeData &md, StateAp *destState, StateAp *srcState ); + void mergeStates( MergeData &md, StateAp *destState, StateAp *srcState ); + + /* Make all states that are combinations of other states and that + * have not yet had their out transitions filled in. This will + * empty out stateDict and stFil. */ + void fillInStates( MergeData &md ); + + /* + * Transition Comparison. + */ + + /* Compare transition data. Either of the pointers may be null. */ + static inline int compareDataPtr( TransAp *trans1, TransAp *trans2 ); + + /* Compare target state and transition data. Either pointer may be null. */ + static inline int compareFullPtr( TransAp *trans1, TransAp *trans2 ); + + /* Compare target partitions. Either pointer may be null. */ + static inline int comparePartPtr( TransAp *trans1, TransAp *trans2 ); + + /* Check marked status of target states. Either pointer may be null. */ + static inline bool shouldMarkPtr( MarkIndex &markIndex, + TransAp *trans1, TransAp *trans2 ); + + /* + * Callbacks. + */ + + /* Compare priority and function table of transitions. */ + static int compareTransData( TransAp *trans1, TransAp *trans2 ); + + /* Add in the properties of srcTrans into this. */ + void addInTrans( TransAp *destTrans, TransAp *srcTrans ); + + /* Compare states on data stored in the states. */ + static int compareStateData( const StateAp *state1, const StateAp *state2 ); + + /* Out transition data. */ + void clearOutData( StateAp *state ); + bool hasOutData( StateAp *state ); + void transferOutData( StateAp *destState, StateAp *srcState ); + + /* + * Allocation. + */ + + /* New up a state and add it to the graph. */ + StateAp *addState(); + + /* + * Building basic machines + */ + + void concatFsm( Key c ); + void concatFsm( Key *str, int len ); + void concatFsmCI( Key *str, int len ); + void orFsm( Key *set, int len ); + void rangeFsm( Key low, Key high ); + void rangeStarFsm( Key low, Key high ); + void emptyFsm( ); + void lambdaFsm( ); + + /* + * Fsm operators. + */ + + void starOp( ); + void repeatOp( int times ); + void optionalRepeatOp( int times ); + void concatOp( FsmAp *other ); + void unionOp( FsmAp *other ); + void intersectOp( FsmAp *other ); + void subtractOp( FsmAp *other ); + void epsilonOp(); + void joinOp( int startId, int finalId, FsmAp **others, int numOthers ); + void globOp( FsmAp **others, int numOthers ); + void deterministicEntry(); + + /* + * Operator workers + */ + + /* Determine if there are any entry points into a start state other than + * the start state. */ + bool isStartStateIsolated(); + + /* Make a new start state that has no entry points. Will not change the + * identity of the fsm. */ + void isolateStartState(); + + /* Workers for resolving epsilon transitions. */ + bool inEptVect( EptVect *eptVect, StateAp *targ ); + void epsilonFillEptVectFrom( StateAp *root, StateAp *from, bool parentLeaving ); + void resolveEpsilonTrans( MergeData &md ); + + /* Workers for concatenation and union. */ + void doConcat( FsmAp *other, StateSet *fromStates, bool optional ); + void doOr( FsmAp *other ); + + /* + * Final states + */ + + /* Unset any final states that are no longer to be final + * due to final bits. */ + void unsetIncompleteFinals(); + void unsetKilledFinals(); + + /* Bring in other's entry points. Assumes others states are going to be + * copied into this machine. */ + void copyInEntryPoints( FsmAp *other ); + + /* Ordering states. */ + void depthFirstOrdering( StateAp *state ); + void depthFirstOrdering(); + void sortStatesByFinal(); + + /* Set sqequential state numbers starting at 0. */ + void setStateNumbers( int base ); + + /* Unset all final states. */ + void unsetAllFinStates(); + + /* Set the bits of final states and clear the bits of non final states. */ + void setFinBits( int finStateBits ); + + /* + * Self-consistency checks. + */ + + /* Run a sanity check on the machine. */ + void verifyIntegrity(); + + /* Verify that there are no unreachable states, or dead end states. */ + void verifyReachability(); + void verifyNoDeadEndStates(); + + /* + * Path pruning + */ + + /* Mark all states reachable from state. */ + void markReachableFromHereReverse( StateAp *state ); + + /* Mark all states reachable from state. */ + void markReachableFromHere( StateAp *state ); + void markReachableFromHereStopFinal( StateAp *state ); + + /* Removes states that cannot be reached by any path in the fsm and are + * thus wasted silicon. */ + void removeDeadEndStates(); + + /* Removes states that cannot be reached by any path in the fsm and are + * thus wasted silicon. */ + void removeUnreachableStates(); + + /* Remove error actions from states on which the error transition will + * never be taken. */ + bool outListCovers( StateAp *state ); + bool anyErrorRange( StateAp *state ); + + /* Remove states that are on the misfit list. */ + void removeMisfits(); + + /* + * FSM Minimization + */ + + /* Minimization by partitioning. */ + void minimizePartition1(); + void minimizePartition2(); + + /* Minimize the final state Machine. The result is the minimal fsm. Slow + * but stable, correct minimization. Uses n^2 space (lookout) and average + * n^2 time. Worst case n^3 time, but a that is a very rare case. */ + void minimizeStable(); + + /* Minimize the final state machine. Does not find the minimal fsm, but a + * pretty good approximation. Does not use any extra space. Average n^2 + * time. Worst case n^3 time, but a that is a very rare case. */ + void minimizeApproximate(); + + /* This is the worker for the minimize approximate solution. It merges + * states that have identical out transitions. */ + bool minimizeRound( ); + + /* Given an intial partioning of states, split partitions that have out trans + * to differing partitions. */ + int partitionRound( StateAp **statePtrs, MinPartition *parts, int numParts ); + + /* Split partitions that have a transition to a previously split partition, until + * there are no more partitions to split. */ + int splitCandidates( StateAp **statePtrs, MinPartition *parts, int numParts ); + + /* Fuse together states in the same partition. */ + void fusePartitions( MinPartition *parts, int numParts ); + + /* Mark pairs where out final stateness differs, out trans data differs, + * trans pairs go to a marked pair or trans data differs. Should get + * alot of pairs. */ + void initialMarkRound( MarkIndex &markIndex ); + + /* One marking round on all state pairs. Considers if trans pairs go + * to a marked state only. Returns whether or not a pair was marked. */ + bool markRound( MarkIndex &markIndex ); + + /* Move the in trans into src into dest. */ + void inTransMove(StateAp *dest, StateAp *src); + + /* Make state src and dest the same state. */ + void fuseEquivStates(StateAp *dest, StateAp *src); + + /* Find any states that didn't get marked by the marking algorithm and + * merge them into the primary states of their equivalence class. */ + void fuseUnmarkedPairs( MarkIndex &markIndex ); + + /* Merge neighboring transitions go to the same state and have the same + * transitions data. */ + void compressTransitions(); + + /* Returns true if there is a transtion (either explicit or by a gap) to + * the error state. */ + bool checkErrTrans( StateAp *state, TransAp *trans ); + bool checkErrTransFinish( StateAp *state ); + bool hasErrorTrans(); + + /* Check if a machine defines a single character. This is useful in + * validating ranges and machines to export. */ + bool checkSingleCharMachine( ); +}; + +#endif diff --git a/contrib/tools/ragel6/fsmmin.cpp b/contrib/tools/ragel6/fsmmin.cpp new file mode 100644 index 0000000000..42ebfaa83a --- /dev/null +++ b/contrib/tools/ragel6/fsmmin.cpp @@ -0,0 +1,732 @@ +/* + * Copyright 2002 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "fsmgraph.h" +#include "mergesort.h" + +int FsmAp::partitionRound( StateAp **statePtrs, MinPartition *parts, int numParts ) +{ + /* Need a mergesort object and a single partition compare. */ + MergeSort<StateAp*, PartitionCompare> mergeSort; + PartitionCompare partCompare; + + /* For each partition. */ + for ( int p = 0; p < numParts; p++ ) { + /* Fill the pointer array with the states in the partition. */ + StateList::Iter state = parts[p].list; + for ( int s = 0; state.lte(); state++, s++ ) + statePtrs[s] = state; + + /* Sort the states using the partitioning compare. */ + int numStates = parts[p].list.length(); + mergeSort.sort( statePtrs, numStates ); + + /* Assign the states into partitions based on the results of the sort. */ + int destPart = p, firstNewPart = numParts; + for ( int s = 1; s < numStates; s++ ) { + /* If this state differs from the last then move to the next partition. */ + if ( partCompare.compare( statePtrs[s-1], statePtrs[s] ) < 0 ) { + /* The new partition is the next avail spot. */ + destPart = numParts; + numParts += 1; + } + + /* If the state is not staying in the first partition, then + * transfer it to its destination partition. */ + if ( destPart != p ) { + StateAp *state = parts[p].list.detach( statePtrs[s] ); + parts[destPart].list.append( state ); + } + } + + /* Fix the partition pointer for all the states that got moved to a new + * partition. This must be done after the states are transfered so the + * result of the sort is not altered. */ + for ( int newPart = firstNewPart; newPart < numParts; newPart++ ) { + StateList::Iter state = parts[newPart].list; + for ( ; state.lte(); state++ ) + state->alg.partition = &parts[newPart]; + } + } + + return numParts; +} + +/** + * \brief Minimize by partitioning version 1. + * + * Repeatedly tries to split partitions until all partitions are unsplittable. + * Produces the most minimal FSM possible. + */ +void FsmAp::minimizePartition1() +{ + /* Need one mergesort object and partition compares. */ + MergeSort<StateAp*, InitPartitionCompare> mergeSort; + InitPartitionCompare initPartCompare; + + /* Nothing to do if there are no states. */ + if ( stateList.length() == 0 ) + return; + + /* + * First thing is to partition the states by final state status and + * transition functions. This gives us an initial partitioning to work + * with. + */ + + /* Make a array of pointers to states. */ + int numStates = stateList.length(); + StateAp** statePtrs = new StateAp*[numStates]; + + /* Fill up an array of pointers to the states for easy sorting. */ + StateList::Iter state = stateList; + for ( int s = 0; state.lte(); state++, s++ ) + statePtrs[s] = state; + + /* Sort the states using the array of states. */ + mergeSort.sort( statePtrs, numStates ); + + /* An array of lists of states is used to partition the states. */ + MinPartition *parts = new MinPartition[numStates]; + + /* Assign the states into partitions. */ + int destPart = 0; + for ( int s = 0; s < numStates; s++ ) { + /* If this state differs from the last then move to the next partition. */ + if ( s > 0 && initPartCompare.compare( statePtrs[s-1], statePtrs[s] ) < 0 ) { + /* Move to the next partition. */ + destPart += 1; + } + + /* Put the state into its partition. */ + statePtrs[s]->alg.partition = &parts[destPart]; + parts[destPart].list.append( statePtrs[s] ); + } + + /* We just moved all the states from the main list into partitions without + * taking them off the main list. So clean up the main list now. */ + stateList.abandon(); + + /* Split partitions. */ + int numParts = destPart + 1; + while ( true ) { + /* Test all partitions for splitting. */ + int newNum = partitionRound( statePtrs, parts, numParts ); + + /* When no partitions can be split, stop. */ + if ( newNum == numParts ) + break; + + numParts = newNum; + } + + /* Fuse states in the same partition. The states will end up back on the + * main list. */ + fusePartitions( parts, numParts ); + + /* Cleanup. */ + delete[] statePtrs; + delete[] parts; +} + +/* Split partitions that need splittting, decide which partitions might need + * to be split as a result, continue until there are no more that might need + * to be split. */ +int FsmAp::splitCandidates( StateAp **statePtrs, MinPartition *parts, int numParts ) +{ + /* Need a mergesort and a partition compare. */ + MergeSort<StateAp*, PartitionCompare> mergeSort; + PartitionCompare partCompare; + + /* The lists of unsplitable (partList) and splitable partitions. + * Only partitions in the splitable list are check for needing splitting. */ + PartitionList partList, splittable; + + /* Initially, all partitions are born from a split (the initial + * partitioning) and can cause other partitions to be split. So any + * partition with a state with a transition out to another partition is a + * candidate for splitting. This will make every partition except possibly + * partitions of final states split candidates. */ + for ( int p = 0; p < numParts; p++ ) { + /* Assume not active. */ + parts[p].active = false; + + /* Look for a trans out of any state in the partition. */ + for ( StateList::Iter state = parts[p].list; state.lte(); state++ ) { + /* If there is at least one transition out to another state then + * the partition becomes splittable. */ + if ( state->outList.length() > 0 ) { + parts[p].active = true; + break; + } + } + + /* If it was found active then it goes on the splittable list. */ + if ( parts[p].active ) + splittable.append( &parts[p] ); + else + partList.append( &parts[p] ); + } + + /* While there are partitions that are splittable, pull one off and try + * to split it. If it splits, determine which partitions may now be split + * as a result of the newly split partition. */ + while ( splittable.length() > 0 ) { + MinPartition *partition = splittable.detachFirst(); + + /* Fill the pointer array with the states in the partition. */ + StateList::Iter state = partition->list; + for ( int s = 0; state.lte(); state++, s++ ) + statePtrs[s] = state; + + /* Sort the states using the partitioning compare. */ + int numStates = partition->list.length(); + mergeSort.sort( statePtrs, numStates ); + + /* Assign the states into partitions based on the results of the sort. */ + MinPartition *destPart = partition; + int firstNewPart = numParts; + for ( int s = 1; s < numStates; s++ ) { + /* If this state differs from the last then move to the next partition. */ + if ( partCompare.compare( statePtrs[s-1], statePtrs[s] ) < 0 ) { + /* The new partition is the next avail spot. */ + destPart = &parts[numParts]; + numParts += 1; + } + + /* If the state is not staying in the first partition, then + * transfer it to its destination partition. */ + if ( destPart != partition ) { + StateAp *state = partition->list.detach( statePtrs[s] ); + destPart->list.append( state ); + } + } + + /* Fix the partition pointer for all the states that got moved to a new + * partition. This must be done after the states are transfered so the + * result of the sort is not altered. */ + int newPart; + for ( newPart = firstNewPart; newPart < numParts; newPart++ ) { + StateList::Iter state = parts[newPart].list; + for ( ; state.lte(); state++ ) + state->alg.partition = &parts[newPart]; + } + + /* Put the partition we just split and any new partitions that came out + * of the split onto the inactive list. */ + partition->active = false; + partList.append( partition ); + for ( newPart = firstNewPart; newPart < numParts; newPart++ ) { + parts[newPart].active = false; + partList.append( &parts[newPart] ); + } + + if ( destPart == partition ) + continue; + + /* Now determine which partitions are splittable as a result of + * splitting partition by walking the in lists of the states in + * partitions that got split. Partition is the faked first item in the + * loop. */ + MinPartition *causalPart = partition; + newPart = firstNewPart - 1; + while ( newPart < numParts ) { + /* Loop all states in the causal partition. */ + StateList::Iter state = causalPart->list; + for ( ; state.lte(); state++ ) { + /* Walk all transition into the state and put the partition + * that the from state is in onto the splittable list. */ + for ( TransInList::Iter trans = state->inList; trans.lte(); trans++ ) { + MinPartition *fromPart = trans->fromState->alg.partition; + if ( ! fromPart->active ) { + fromPart->active = true; + partList.detach( fromPart ); + splittable.append( fromPart ); + } + } + } + + newPart += 1; + causalPart = &parts[newPart]; + } + } + return numParts; +} + + +/** + * \brief Minimize by partitioning version 2 (best alg). + * + * Repeatedly tries to split partitions that may splittable until there are no + * more partitions that might possibly need splitting. Runs faster than + * version 1. Produces the most minimal fsm possible. + */ +void FsmAp::minimizePartition2() +{ + /* Need a mergesort and an initial partition compare. */ + MergeSort<StateAp*, InitPartitionCompare> mergeSort; + InitPartitionCompare initPartCompare; + + /* Nothing to do if there are no states. */ + if ( stateList.length() == 0 ) + return; + + /* + * First thing is to partition the states by final state status and + * transition functions. This gives us an initial partitioning to work + * with. + */ + + /* Make a array of pointers to states. */ + int numStates = stateList.length(); + StateAp** statePtrs = new StateAp*[numStates]; + + /* Fill up an array of pointers to the states for easy sorting. */ + StateList::Iter state = stateList; + for ( int s = 0; state.lte(); state++, s++ ) + statePtrs[s] = state; + + /* Sort the states using the array of states. */ + mergeSort.sort( statePtrs, numStates ); + + /* An array of lists of states is used to partition the states. */ + MinPartition *parts = new MinPartition[numStates]; + + /* Assign the states into partitions. */ + int destPart = 0; + for ( int s = 0; s < numStates; s++ ) { + /* If this state differs from the last then move to the next partition. */ + if ( s > 0 && initPartCompare.compare( statePtrs[s-1], statePtrs[s] ) < 0 ) { + /* Move to the next partition. */ + destPart += 1; + } + + /* Put the state into its partition. */ + statePtrs[s]->alg.partition = &parts[destPart]; + parts[destPart].list.append( statePtrs[s] ); + } + + /* We just moved all the states from the main list into partitions without + * taking them off the main list. So clean up the main list now. */ + stateList.abandon(); + + /* Split partitions. */ + int numParts = splitCandidates( statePtrs, parts, destPart+1 ); + + /* Fuse states in the same partition. The states will end up back on the + * main list. */ + fusePartitions( parts, numParts ); + + /* Cleanup. */ + delete[] statePtrs; + delete[] parts; +} + +void FsmAp::initialMarkRound( MarkIndex &markIndex ) +{ + /* P and q for walking pairs. */ + StateAp *p = stateList.head, *q; + + /* Need an initial partition compare. */ + InitPartitionCompare initPartCompare; + + /* Walk all unordered pairs of (p, q) where p != q. + * The second depth of the walk stops before reaching p. This + * gives us all unordered pairs of states (p, q) where p != q. */ + while ( p != 0 ) { + q = stateList.head; + while ( q != p ) { + /* If the states differ on final state status, out transitions or + * any transition data then they should be separated on the initial + * round. */ + if ( initPartCompare.compare( p, q ) != 0 ) + markIndex.markPair( p->alg.stateNum, q->alg.stateNum ); + + q = q->next; + } + p = p->next; + } +} + +bool FsmAp::markRound( MarkIndex &markIndex ) +{ + /* P an q for walking pairs. Take note if any pair gets marked. */ + StateAp *p = stateList.head, *q; + bool pairWasMarked = false; + + /* Need a mark comparison. */ + MarkCompare markCompare; + + /* Walk all unordered pairs of (p, q) where p != q. + * The second depth of the walk stops before reaching p. This + * gives us all unordered pairs of states (p, q) where p != q. */ + while ( p != 0 ) { + q = stateList.head; + while ( q != p ) { + /* Should we mark the pair? */ + if ( !markIndex.isPairMarked( p->alg.stateNum, q->alg.stateNum ) ) { + if ( markCompare.shouldMark( markIndex, p, q ) ) { + markIndex.markPair( p->alg.stateNum, q->alg.stateNum ); + pairWasMarked = true; + } + } + q = q->next; + } + p = p->next; + } + + return pairWasMarked; +} + + +/** + * \brief Minimize by pair marking. + * + * Decides if each pair of states is distinct or not. Uses O(n^2) memory and + * should only be used on small graphs. Produces the most minmimal FSM + * possible. + */ +void FsmAp::minimizeStable() +{ + /* Set the state numbers. */ + setStateNumbers( 0 ); + + /* This keeps track of which pairs have been marked. */ + MarkIndex markIndex( stateList.length() ); + + /* Mark pairs where final stateness, out trans, or trans data differ. */ + initialMarkRound( markIndex ); + + /* While the last round of marking succeeded in marking a state + * continue to do another round. */ + int modified = markRound( markIndex ); + while (modified) + modified = markRound( markIndex ); + + /* Merge pairs that are unmarked. */ + fuseUnmarkedPairs( markIndex ); +} + +bool FsmAp::minimizeRound() +{ + /* Nothing to do if there are no states. */ + if ( stateList.length() == 0 ) + return false; + + /* Need a mergesort on approx compare and an approx compare. */ + MergeSort<StateAp*, ApproxCompare> mergeSort; + ApproxCompare approxCompare; + + /* Fill up an array of pointers to the states. */ + StateAp **statePtrs = new StateAp*[stateList.length()]; + StateList::Iter state = stateList; + for ( int s = 0; state.lte(); state++, s++ ) + statePtrs[s] = state; + + bool modified = false; + + /* Sort The list. */ + mergeSort.sort( statePtrs, stateList.length() ); + + /* Walk the list looking for duplicates next to each other, + * merge in any duplicates. */ + StateAp **pLast = statePtrs; + StateAp **pState = statePtrs + 1; + for ( int i = 1; i < stateList.length(); i++, pState++ ) { + if ( approxCompare.compare( *pLast, *pState ) == 0 ) { + /* Last and pState are the same, so fuse together. Move forward + * with pState but not with pLast. If any more are identical, we + * must */ + fuseEquivStates( *pLast, *pState ); + modified = true; + } + else { + /* Last and this are different, do not set to merge them. Move + * pLast to the current (it may be way behind from merging many + * states) and pState forward one to consider the next pair. */ + pLast = pState; + } + } + delete[] statePtrs; + return modified; +} + +/** + * \brief Minmimize by an approximation. + * + * Repeatedly tries to find states with transitions out to the same set of + * states on the same set of keys until no more identical states can be found. + * Does not produce the most minimial FSM possible. + */ +void FsmAp::minimizeApproximate() +{ + /* While the last minimization round succeeded in compacting states, + * continue to try to compact states. */ + while ( true ) { + bool modified = minimizeRound(); + if ( ! modified ) + break; + } +} + + +/* Remove states that have no path to them from the start state. Recursively + * traverses the graph marking states that have paths into them. Then removes + * all states that did not get marked. */ +void FsmAp::removeUnreachableStates() +{ + /* Misfit accounting should be off and there should be no states on the + * misfit list. */ + assert( !misfitAccounting && misfitList.length() == 0 ); + + /* Mark all the states that can be reached + * through the existing set of entry points. */ + markReachableFromHere( startState ); + for ( EntryMap::Iter en = entryPoints; en.lte(); en++ ) + markReachableFromHere( en->value ); + + /* Delete all states that are not marked + * and unmark the ones that are marked. */ + StateAp *state = stateList.head; + while ( state ) { + StateAp *next = state->next; + + if ( state->stateBits & STB_ISMARKED ) + state->stateBits &= ~ STB_ISMARKED; + else { + detachState( state ); + stateList.detach( state ); + delete state; + } + + state = next; + } +} + +bool FsmAp::outListCovers( StateAp *state ) +{ + /* Must be at least one range to cover. */ + if ( state->outList.length() == 0 ) + return false; + + /* The first must start at the lower bound. */ + TransList::Iter trans = state->outList.first(); + if ( keyOps->minKey < trans->lowKey ) + return false; + + /* Loop starts at second el. */ + trans.increment(); + + /* Loop checks lower against prev upper. */ + for ( ; trans.lte(); trans++ ) { + /* Lower end of the trans must be one greater than the + * previous' high end. */ + Key lowKey = trans->lowKey; + lowKey.decrement(); + if ( trans->prev->highKey < lowKey ) + return false; + } + + /* Require that the last range extends to the upper bound. */ + trans = state->outList.last(); + if ( trans->highKey < keyOps->maxKey ) + return false; + + return true; +} + +/* Remove states that that do not lead to a final states. Works recursivly traversing + * the graph in reverse (starting from all final states) and marking seen states. Then + * removes states that did not get marked. */ +void FsmAp::removeDeadEndStates() +{ + /* Misfit accounting should be off and there should be no states on the + * misfit list. */ + assert( !misfitAccounting && misfitList.length() == 0 ); + + /* Mark all states that have paths to the final states. */ + StateAp **st = finStateSet.data; + int nst = finStateSet.length(); + for ( int i = 0; i < nst; i++, st++ ) + markReachableFromHereReverse( *st ); + + /* Start state gets honorary marking. If the machine accepts nothing we + * still want the start state to hang around. This must be done after the + * recursive call on all the final states so that it does not cause the + * start state in transitions to be skipped when the start state is + * visited by the traversal. */ + startState->stateBits |= STB_ISMARKED; + + /* Delete all states that are not marked + * and unmark the ones that are marked. */ + StateAp *state = stateList.head; + while ( state != 0 ) { + StateAp *next = state->next; + + if ( state->stateBits & STB_ISMARKED ) + state->stateBits &= ~ STB_ISMARKED; + else { + detachState( state ); + stateList.detach( state ); + delete state; + } + + state = next; + } +} + +/* Remove states on the misfit list. To work properly misfit accounting should + * be on when this is called. The detaching of a state will likely cause + * another misfit to be collected and it can then be removed. */ +void FsmAp::removeMisfits() +{ + while ( misfitList.length() > 0 ) { + /* Get the first state. */ + StateAp *state = misfitList.head; + + /* Detach and delete. */ + detachState( state ); + + /* The state was previously on the misfit list and detaching can only + * remove in transitions so the state must still be on the misfit + * list. */ + misfitList.detach( state ); + delete state; + } +} + +/* Fuse src into dest because they have been deemed equivalent states. + * Involves moving transitions into src to go into dest and invoking + * callbacks. Src is deleted detached from the graph and deleted. */ +void FsmAp::fuseEquivStates( StateAp *dest, StateAp *src ) +{ + /* This would get ugly. */ + assert( dest != src ); + + /* Cur is a duplicate. We can merge it with trail. */ + inTransMove( dest, src ); + + detachState( src ); + stateList.detach( src ); + delete src; +} + +void FsmAp::fuseUnmarkedPairs( MarkIndex &markIndex ) +{ + StateAp *p = stateList.head, *nextP, *q; + + /* Definition: The primary state of an equivalence class is the first state + * encounterd that belongs to the equivalence class. All equivalence + * classes have primary state including equivalence classes with one state + * in it. */ + + /* For each unmarked pair merge p into q and delete p. q is always the + * primary state of it's equivalence class. We wouldn't have landed on it + * here if it were not, because it would have been deleted. + * + * Proof that q is the primaray state of it's equivalence class: Assume q + * is not the primary state of it's equivalence class, then it would be + * merged into some state that came before it and thus p would be + * equivalent to that state. But q is the first state that p is equivalent + * to so we have a contradiction. */ + + /* Walk all unordered pairs of (p, q) where p != q. + * The second depth of the walk stops before reaching p. This + * gives us all unordered pairs of states (p, q) where p != q. */ + while ( p != 0 ) { + nextP = p->next; + + q = stateList.head; + while ( q != p ) { + /* If one of p or q is a final state then mark. */ + if ( ! markIndex.isPairMarked( p->alg.stateNum, q->alg.stateNum ) ) { + fuseEquivStates( q, p ); + break; + } + q = q->next; + } + p = nextP; + } +} + +void FsmAp::fusePartitions( MinPartition *parts, int numParts ) +{ + /* For each partition, fuse state 2, 3, ... into state 1. */ + for ( int p = 0; p < numParts; p++ ) { + /* Assume that there will always be at least one state. */ + StateAp *first = parts[p].list.head, *toFuse = first->next; + + /* Put the first state back onto the main state list. Don't bother + * removing it from the partition list first. */ + stateList.append( first ); + + /* Fuse the rest of the state into the first. */ + while ( toFuse != 0 ) { + /* Save the next. We will trash it before it is needed. */ + StateAp *next = toFuse->next; + + /* Put the state to be fused in to the first back onto the main + * list before it is fuse. the graph. The state needs to be on + * the main list for the detach from the graph to work. Don't + * bother removing the state from the partition list first. We + * need not maintain it. */ + stateList.append( toFuse ); + + /* Now fuse to the first. */ + fuseEquivStates( first, toFuse ); + + /* Go to the next that we saved before trashing the next pointer. */ + toFuse = next; + } + + /* We transfered the states from the partition list into the main list without + * removing the states from the partition list first. Clean it up. */ + parts[p].list.abandon(); + } +} + + +/* Merge neighboring transitions go to the same state and have the same + * transitions data. */ +void FsmAp::compressTransitions() +{ + for ( StateList::Iter st = stateList; st.lte(); st++ ) { + if ( st->outList.length() > 1 ) { + for ( TransList::Iter trans = st->outList, next = trans.next(); next.lte(); ) { + Key nextLow = next->lowKey; + nextLow.decrement(); + if ( trans->highKey == nextLow && trans->toState == next->toState && + CmpActionTable::compare( trans->actionTable, next->actionTable ) == 0 ) + { + trans->highKey = next->highKey; + st->outList.detach( next ); + detachTrans( next->fromState, next->toState, next ); + delete next; + next = trans.next(); + } + else { + trans.increment(); + next.increment(); + } + } + } + } +} diff --git a/contrib/tools/ragel6/fsmstate.cpp b/contrib/tools/ragel6/fsmstate.cpp new file mode 100644 index 0000000000..63c48e96ab --- /dev/null +++ b/contrib/tools/ragel6/fsmstate.cpp @@ -0,0 +1,490 @@ +/* + * Copyright 2002 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <string.h> +#include <assert.h> +#include "fsmgraph.h" + +#include <iostream> +using namespace std; + +/* Construct a mark index for a specified number of states. Must new up + * an array that is states^2 in size. */ +MarkIndex::MarkIndex( int states ) : numStates(states) +{ + /* Total pairs is states^2. Actually only use half of these, but we allocate + * them all to make indexing into the array easier. */ + int total = states * states; + + /* New up chars so that individual DListEl constructors are + * not called. Zero out the mem manually. */ + array = new bool[total]; + memset( array, 0, sizeof(bool) * total ); +} + +/* Free the array used to store state pairs. */ +MarkIndex::~MarkIndex() +{ + delete[] array; +} + +/* Mark a pair of states. States are specified by their number. The + * marked states are moved from the unmarked list to the marked list. */ +void MarkIndex::markPair(int state1, int state2) +{ + int pos = ( state1 >= state2 ) ? + ( state1 * numStates ) + state2 : + ( state2 * numStates ) + state1; + + array[pos] = true; +} + +/* Returns true if the pair of states are marked. Returns false otherwise. + * Ordering of states given does not matter. */ +bool MarkIndex::isPairMarked(int state1, int state2) +{ + int pos = ( state1 >= state2 ) ? + ( state1 * numStates ) + state2 : + ( state2 * numStates ) + state1; + + return array[pos]; +} + +/* Create a new fsm state. State has not out transitions or in transitions, not + * out out transition data and not number. */ +StateAp::StateAp() +: + /* No out or in transitions. */ + outList(), + inList(), + + /* No EOF target. */ + eofTarget(0), + + /* No entry points, or epsilon trans. */ + entryIds(), + epsilonTrans(), + + /* Conditions. */ + stateCondList(), + + /* No transitions in from other states. */ + foreignInTrans(0), + + /* Only used during merging. Normally null. */ + stateDictEl(0), + eptVect(0), + + /* No state identification bits. */ + stateBits(0), + + /* No Priority data. */ + outPriorTable(), + + /* No Action data. */ + toStateActionTable(), + fromStateActionTable(), + outActionTable(), + outCondSet(), + errActionTable(), + eofActionTable() +{ +} + +/* Copy everything except actual the transitions. That is left up to the + * FsmAp copy constructor. */ +StateAp::StateAp(const StateAp &other) +: + /* All lists are cleared. They will be filled in when the + * individual transitions are duplicated and attached. */ + outList(), + inList(), + + /* Set this using the original state's eofTarget. It will get mapped back + * to the new machine in the Fsm copy constructor. */ + eofTarget(other.eofTarget), + + /* Duplicate the entry id set and epsilon transitions. These + * are sets of integers and as such need no fixing. */ + entryIds(other.entryIds), + epsilonTrans(other.epsilonTrans), + + /* Copy in the elements of the conditions. */ + stateCondList( other.stateCondList ), + + /* No transitions in from other states. */ + foreignInTrans(0), + + /* This is only used during merging. Normally null. */ + stateDictEl(0), + eptVect(0), + + /* Fsm state data. */ + stateBits(other.stateBits), + + /* Copy in priority data. */ + outPriorTable(other.outPriorTable), + + /* Copy in action data. */ + toStateActionTable(other.toStateActionTable), + fromStateActionTable(other.fromStateActionTable), + outActionTable(other.outActionTable), + outCondSet(other.outCondSet), + errActionTable(other.errActionTable), + eofActionTable(other.eofActionTable) +{ + /* Duplicate all the transitions. */ + for ( TransList::Iter trans = other.outList; trans.lte(); trans++ ) { + /* Dupicate and store the orginal target in the transition. This will + * be corrected once all the states have been created. */ + TransAp *newTrans = new TransAp(*trans); + assert( trans->lmActionTable.length() == 0 ); + newTrans->toState = trans->toState; + outList.append( newTrans ); + } +} + +/* If there is a state dict element, then delete it. Everything else is left + * up to the FsmGraph destructor. */ +StateAp::~StateAp() +{ + if ( stateDictEl != 0 ) + delete stateDictEl; +} + +/* Compare two states using pointers to the states. With the approximate + * compare, the idea is that if the compare finds them the same, they can + * immediately be merged. */ +int ApproxCompare::compare( const StateAp *state1, const StateAp *state2 ) +{ + int compareRes; + + /* Test final state status. */ + if ( (state1->stateBits & STB_ISFINAL) && !(state2->stateBits & STB_ISFINAL) ) + return -1; + else if ( !(state1->stateBits & STB_ISFINAL) && (state2->stateBits & STB_ISFINAL) ) + return 1; + + /* Test epsilon transition sets. */ + compareRes = CmpEpsilonTrans::compare( state1->epsilonTrans, + state2->epsilonTrans ); + if ( compareRes != 0 ) + return compareRes; + + /* Compare the out transitions. */ + compareRes = FsmAp::compareStateData( state1, state2 ); + if ( compareRes != 0 ) + return compareRes; + + /* Use a pair iterator to get the transition pairs. */ + PairIter<TransAp> outPair( state1->outList.head, state2->outList.head ); + for ( ; !outPair.end(); outPair++ ) { + switch ( outPair.userState ) { + + case RangeInS1: + compareRes = FsmAp::compareFullPtr( outPair.s1Tel.trans, 0 ); + if ( compareRes != 0 ) + return compareRes; + break; + + case RangeInS2: + compareRes = FsmAp::compareFullPtr( 0, outPair.s2Tel.trans ); + if ( compareRes != 0 ) + return compareRes; + break; + + case RangeOverlap: + compareRes = FsmAp::compareFullPtr( + outPair.s1Tel.trans, outPair.s2Tel.trans ); + if ( compareRes != 0 ) + return compareRes; + break; + + case BreakS1: + case BreakS2: + break; + } + } + + /* Check EOF targets. */ + if ( state1->eofTarget < state2->eofTarget ) + return -1; + else if ( state1->eofTarget > state2->eofTarget ) + return 1; + + /* Got through the entire state comparison, deem them equal. */ + return 0; +} + +/* Compare class used in the initial partition. */ +int InitPartitionCompare::compare( const StateAp *state1 , const StateAp *state2 ) +{ + int compareRes; + + /* Test final state status. */ + if ( (state1->stateBits & STB_ISFINAL) && !(state2->stateBits & STB_ISFINAL) ) + return -1; + else if ( !(state1->stateBits & STB_ISFINAL) && (state2->stateBits & STB_ISFINAL) ) + return 1; + + /* Test epsilon transition sets. */ + compareRes = CmpEpsilonTrans::compare( state1->epsilonTrans, + state2->epsilonTrans ); + if ( compareRes != 0 ) + return compareRes; + + /* Compare the out transitions. */ + compareRes = FsmAp::compareStateData( state1, state2 ); + if ( compareRes != 0 ) + return compareRes; + + /* Use a pair iterator to test the condition pairs. */ + PairIter<StateCond> condPair( state1->stateCondList.head, state2->stateCondList.head ); + for ( ; !condPair.end(); condPair++ ) { + switch ( condPair.userState ) { + case RangeInS1: + return 1; + case RangeInS2: + return -1; + + case RangeOverlap: { + CondSpace *condSpace1 = condPair.s1Tel.trans->condSpace; + CondSpace *condSpace2 = condPair.s2Tel.trans->condSpace; + if ( condSpace1 < condSpace2 ) + return -1; + else if ( condSpace1 > condSpace2 ) + return 1; + break; + } + case BreakS1: + case BreakS2: + break; + } + } + + /* Use a pair iterator to test the transition pairs. */ + PairIter<TransAp> outPair( state1->outList.head, state2->outList.head ); + for ( ; !outPair.end(); outPair++ ) { + switch ( outPair.userState ) { + + case RangeInS1: + compareRes = FsmAp::compareDataPtr( outPair.s1Tel.trans, 0 ); + if ( compareRes != 0 ) + return compareRes; + break; + + case RangeInS2: + compareRes = FsmAp::compareDataPtr( 0, outPair.s2Tel.trans ); + if ( compareRes != 0 ) + return compareRes; + break; + + case RangeOverlap: + compareRes = FsmAp::compareDataPtr( + outPair.s1Tel.trans, outPair.s2Tel.trans ); + if ( compareRes != 0 ) + return compareRes; + break; + + case BreakS1: + case BreakS2: + break; + } + } + + return 0; +} + +/* Compare class for the sort that does the partitioning. */ +int PartitionCompare::compare( const StateAp *state1, const StateAp *state2 ) +{ + int compareRes; + + /* Use a pair iterator to get the transition pairs. */ + PairIter<TransAp> outPair( state1->outList.head, state2->outList.head ); + for ( ; !outPair.end(); outPair++ ) { + switch ( outPair.userState ) { + + case RangeInS1: + compareRes = FsmAp::comparePartPtr( outPair.s1Tel.trans, 0 ); + if ( compareRes != 0 ) + return compareRes; + break; + + case RangeInS2: + compareRes = FsmAp::comparePartPtr( 0, outPair.s2Tel.trans ); + if ( compareRes != 0 ) + return compareRes; + break; + + case RangeOverlap: + compareRes = FsmAp::comparePartPtr( + outPair.s1Tel.trans, outPair.s2Tel.trans ); + if ( compareRes != 0 ) + return compareRes; + break; + + case BreakS1: + case BreakS2: + break; + } + } + + /* Test eof targets. */ + if ( state1->eofTarget == 0 && state2->eofTarget != 0 ) + return -1; + else if ( state1->eofTarget != 0 && state2->eofTarget == 0 ) + return 1; + else if ( state1->eofTarget != 0 ) { + /* Both eof targets are set. */ + compareRes = CmpOrd< MinPartition* >::compare( + state1->eofTarget->alg.partition, state2->eofTarget->alg.partition ); + if ( compareRes != 0 ) + return compareRes; + } + + return 0; +} + +/* Compare class for the sort that does the partitioning. */ +bool MarkCompare::shouldMark( MarkIndex &markIndex, const StateAp *state1, + const StateAp *state2 ) +{ + /* Use a pair iterator to get the transition pairs. */ + PairIter<TransAp> outPair( state1->outList.head, state2->outList.head ); + for ( ; !outPair.end(); outPair++ ) { + switch ( outPair.userState ) { + + case RangeInS1: + if ( FsmAp::shouldMarkPtr( markIndex, outPair.s1Tel.trans, 0 ) ) + return true; + break; + + case RangeInS2: + if ( FsmAp::shouldMarkPtr( markIndex, 0, outPair.s2Tel.trans ) ) + return true; + break; + + case RangeOverlap: + if ( FsmAp::shouldMarkPtr( markIndex, + outPair.s1Tel.trans, outPair.s2Tel.trans ) ) + return true; + break; + + case BreakS1: + case BreakS2: + break; + } + } + + return false; +} + +/* + * Transition Comparison. + */ + +/* Compare target partitions. Either pointer may be null. */ +int FsmAp::comparePartPtr( TransAp *trans1, TransAp *trans2 ) +{ + if ( trans1 != 0 ) { + /* If trans1 is set then so should trans2. The initial partitioning + * guarantees this for us. */ + if ( trans1->toState == 0 && trans2->toState != 0 ) + return -1; + else if ( trans1->toState != 0 && trans2->toState == 0 ) + return 1; + else if ( trans1->toState != 0 ) { + /* Both of targets are set. */ + return CmpOrd< MinPartition* >::compare( + trans1->toState->alg.partition, trans2->toState->alg.partition ); + } + } + return 0; +} + + +/* Compares two transition pointers according to priority and functions. + * Either pointer may be null. Does not consider to state or from state. */ +int FsmAp::compareDataPtr( TransAp *trans1, TransAp *trans2 ) +{ + if ( trans1 == 0 && trans2 != 0 ) + return -1; + else if ( trans1 != 0 && trans2 == 0 ) + return 1; + else if ( trans1 != 0 ) { + /* Both of the transition pointers are set. */ + int compareRes = compareTransData( trans1, trans2 ); + if ( compareRes != 0 ) + return compareRes; + } + return 0; +} + +/* Compares two transitions according to target state, priority and functions. + * Does not consider from state. Either of the pointers may be null. */ +int FsmAp::compareFullPtr( TransAp *trans1, TransAp *trans2 ) +{ + if ( (trans1 != 0) ^ (trans2 != 0) ) { + /* Exactly one of the transitions is set. */ + if ( trans1 != 0 ) + return -1; + else + return 1; + } + else if ( trans1 != 0 ) { + /* Both of the transition pointers are set. Test target state, + * priority and funcs. */ + if ( trans1->toState < trans2->toState ) + return -1; + else if ( trans1->toState > trans2->toState ) + return 1; + else if ( trans1->toState != 0 ) { + /* Test transition data. */ + int compareRes = compareTransData( trans1, trans2 ); + if ( compareRes != 0 ) + return compareRes; + } + } + return 0; +} + + +bool FsmAp::shouldMarkPtr( MarkIndex &markIndex, TransAp *trans1, + TransAp *trans2 ) +{ + if ( (trans1 != 0) ^ (trans2 != 0) ) { + /* Exactly one of the transitions is set. The initial mark round + * should rule out this case. */ + assert( false ); + } + else if ( trans1 != 0 ) { + /* Both of the transitions are set. If the target pair is marked, then + * the pair we are considering gets marked. */ + return markIndex.isPairMarked( trans1->toState->alg.stateNum, + trans2->toState->alg.stateNum ); + } + + /* Neither of the transitiosn are set. */ + return false; +} + + diff --git a/contrib/tools/ragel6/gendata.cpp b/contrib/tools/ragel6/gendata.cpp new file mode 100644 index 0000000000..737a53c7d8 --- /dev/null +++ b/contrib/tools/ragel6/gendata.cpp @@ -0,0 +1,1166 @@ +/* + * Copyright 2005-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "gendata.h" +#include "ragel.h" +#include <iostream> + +/* + * Code generators. + */ + +#include "cstable.h" +#include "csftable.h" +#include "csflat.h" +#include "csfflat.h" +#include "csgoto.h" +#include "csfgoto.h" +#include "csipgoto.h" +#include "cssplit.h" + +#include "cdtable.h" +#include "cdftable.h" +#include "cdflat.h" +#include "cdfflat.h" +#include "cdgoto.h" +#include "cdfgoto.h" +#include "cdipgoto.h" +#include "cdsplit.h" + +#include "dotcodegen.h" + +#include "javacodegen.h" + +#include "gocodegen.h" +#include "gotable.h" +#include "goftable.h" +#include "goflat.h" +#include "gofflat.h" +#include "gogoto.h" +#include "gofgoto.h" +#include "goipgoto.h" + +#include "mltable.h" +#include "mlftable.h" +#include "mlflat.h" +#include "mlfflat.h" +#include "mlgoto.h" +#include "mlfgoto.h" + +#include "rubytable.h" +#include "rubyftable.h" +#include "rubyflat.h" +#include "rubyfflat.h" +#include "rbxgoto.h" + +string itoa( int i ) +{ + char buf[16]; + sprintf( buf, "%i", i ); + return buf; +} + +using std::cout; +using std::cerr; +using std::endl; + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *dotMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = new GraphvizDotGen(out); + + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + /* For normal code generation we want a transition on every character so we never + * end up in an undefined state. For graphviz this just clutters the + * drawing so we turn it off. */ + codeGen->wantComplete = false; + + return codeGen; +} + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *cdMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = 0; + switch ( hostLang->lang ) { + case HostLang::C: + switch ( codeStyle ) { + case GenTables: + codeGen = new CTabCodeGen(out); + break; + case GenFTables: + codeGen = new CFTabCodeGen(out); + break; + case GenFlat: + codeGen = new CFlatCodeGen(out); + break; + case GenFFlat: + codeGen = new CFFlatCodeGen(out); + break; + case GenGoto: + codeGen = new CGotoCodeGen(out); + break; + case GenFGoto: + codeGen = new CFGotoCodeGen(out); + break; + case GenIpGoto: + codeGen = new CIpGotoCodeGen(out); + break; + case GenSplit: + codeGen = new CSplitCodeGen(out); + break; + } + break; + + case HostLang::D: + switch ( codeStyle ) { + case GenTables: + codeGen = new DTabCodeGen(out); + break; + case GenFTables: + codeGen = new DFTabCodeGen(out); + break; + case GenFlat: + codeGen = new DFlatCodeGen(out); + break; + case GenFFlat: + codeGen = new DFFlatCodeGen(out); + break; + case GenGoto: + codeGen = new DGotoCodeGen(out); + break; + case GenFGoto: + codeGen = new DFGotoCodeGen(out); + break; + case GenIpGoto: + codeGen = new DIpGotoCodeGen(out); + break; + case GenSplit: + codeGen = new DSplitCodeGen(out); + break; + } + break; + + case HostLang::D2: + switch ( codeStyle ) { + case GenTables: + codeGen = new D2TabCodeGen(out); + break; + case GenFTables: + codeGen = new D2FTabCodeGen(out); + break; + case GenFlat: + codeGen = new D2FlatCodeGen(out); + break; + case GenFFlat: + codeGen = new D2FFlatCodeGen(out); + break; + case GenGoto: + codeGen = new D2GotoCodeGen(out); + break; + case GenFGoto: + codeGen = new D2FGotoCodeGen(out); + break; + case GenIpGoto: + codeGen = new D2IpGotoCodeGen(out); + break; + case GenSplit: + codeGen = new D2SplitCodeGen(out); + break; + } + break; + + default: break; + } + + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + return codeGen; +} + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *javaMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = new JavaTabCodeGen(out); + + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + return codeGen; +} + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *goMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = 0; + + switch ( codeStyle ) { + case GenTables: + codeGen = new GoTabCodeGen(out); + break; + case GenFTables: + codeGen = new GoFTabCodeGen(out); + break; + case GenFlat: + codeGen = new GoFlatCodeGen(out); + break; + case GenFFlat: + codeGen = new GoFFlatCodeGen(out); + break; + case GenGoto: + codeGen = new GoGotoCodeGen(out); + break; + case GenFGoto: + codeGen = new GoFGotoCodeGen(out); + break; + case GenIpGoto: + codeGen = new GoIpGotoCodeGen(out); + break; + default: + cerr << "Invalid output style, only -T0, -T1, -F0, -F1, -G0, -G1 and -G2 are supported for Go.\n"; + exit(1); + } + + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + return codeGen; +} + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *rubyMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = 0; + switch ( codeStyle ) { + case GenTables: + codeGen = new RubyTabCodeGen(out); + break; + case GenFTables: + codeGen = new RubyFTabCodeGen(out); + break; + case GenFlat: + codeGen = new RubyFlatCodeGen(out); + break; + case GenFFlat: + codeGen = new RubyFFlatCodeGen(out); + break; + case GenGoto: + if ( rubyImpl == Rubinius ) { + codeGen = new RbxGotoCodeGen(out); + } else { + cerr << "Goto style is still _very_ experimental " + "and only supported using Rubinius.\n" + "You may want to enable the --rbx flag " + " to give it a try.\n"; + exit(1); + } + break; + default: + cout << "Invalid code style\n"; + exit(1); + break; + } + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + return codeGen; +} + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *csharpMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = 0; + + switch ( codeStyle ) { + case GenTables: + codeGen = new CSharpTabCodeGen(out); + break; + case GenFTables: + codeGen = new CSharpFTabCodeGen(out); + break; + case GenFlat: + codeGen = new CSharpFlatCodeGen(out); + break; + case GenFFlat: + codeGen = new CSharpFFlatCodeGen(out); + break; + case GenGoto: + codeGen = new CSharpGotoCodeGen(out); + break; + case GenFGoto: + codeGen = new CSharpFGotoCodeGen(out); + break; + case GenIpGoto: + codeGen = new CSharpIpGotoCodeGen(out); + break; + case GenSplit: + codeGen = new CSharpSplitCodeGen(out); + break; + } + + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + return codeGen; +} + +/* Invoked by the parser when a ragel definition is opened. */ +CodeGenData *ocamlMakeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *codeGen = 0; + + switch ( codeStyle ) { + case GenTables: + codeGen = new OCamlTabCodeGen(out); + break; + case GenFTables: + codeGen = new OCamlFTabCodeGen(out); + break; + case GenFlat: + codeGen = new OCamlFlatCodeGen(out); + break; + case GenFFlat: + codeGen = new OCamlFFlatCodeGen(out); + break; + case GenGoto: + codeGen = new OCamlGotoCodeGen(out); + break; + case GenFGoto: + codeGen = new OCamlFGotoCodeGen(out); + break; + default: + cerr << "I only support the -T0 -T1 -F0 -F1 -G0 and -G1 output styles for OCaml.\n"; + exit(1); + } + + codeGen->sourceFileName = sourceFileName; + codeGen->fsmName = fsmName; + + return codeGen; +} + + +CodeGenData *makeCodeGen( const char *sourceFileName, const char *fsmName, ostream &out ) +{ + CodeGenData *cgd = 0; + if ( generateDot ) + cgd = dotMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangC ) + cgd = cdMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangD ) + cgd = cdMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangD2 ) + cgd = cdMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangGo ) + cgd = goMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangJava ) + cgd = javaMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangRuby ) + cgd = rubyMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangCSharp ) + cgd = csharpMakeCodeGen( sourceFileName, fsmName, out ); + else if ( hostLang == &hostLangOCaml ) + cgd = ocamlMakeCodeGen( sourceFileName, fsmName, out ); + return cgd; +} + +void lineDirective( ostream &out, const char *fileName, int line ) +{ + if ( !generateDot ) { + if ( hostLang == &hostLangC ) + cdLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangD ) + cdLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangD2 ) + cdLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangGo ) + goLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangJava ) + javaLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangRuby ) + rubyLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangCSharp ) + csharpLineDirective( out, fileName, line ); + else if ( hostLang == &hostLangOCaml ) + ocamlLineDirective( out, fileName, line ); + } +} + +void genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + lineDirective( out, filter->fileName, filter->line + 1 ); +} + + +/* Total error count. */ +/* int gblErrorCount = 0; */ + +CodeGenData::CodeGenData( ostream &out ) +: + sourceFileName(0), + fsmName(0), + out(out), + redFsm(0), + allActions(0), + allActionTables(0), + allConditions(0), + allCondSpaces(0), + allStates(0), + nameIndex(0), + startState(-1), + errState(-1), + getKeyExpr(0), + accessExpr(0), + prePushExpr(0), + postPopExpr(0), + pExpr(0), + peExpr(0), + eofExpr(0), + csExpr(0), + topExpr(0), + stackExpr(0), + actExpr(0), + tokstartExpr(0), + tokendExpr(0), + dataExpr(0), + wantComplete(true), + hasLongestMatch(false), + noEnd(false), + noPrefix(false), + noFinal(false), + noError(false), + noEntry(false), + noCS(false) +{} + + +void CodeGenData::createMachine() +{ + redFsm = new RedFsmAp(); +} + +void CodeGenData::initActionList( unsigned long length ) +{ + allActions = new GenAction[length]; + for ( unsigned long a = 0; a < length; a++ ) + actionList.append( allActions+a ); +} + +void CodeGenData::newAction( int anum, const char *name, + const InputLoc &loc, GenInlineList *inlineList ) +{ + allActions[anum].actionId = anum; + allActions[anum].name = name; + allActions[anum].loc = loc; + allActions[anum].inlineList = inlineList; +} + +void CodeGenData::initActionTableList( unsigned long length ) +{ + allActionTables = new RedAction[length]; +} + +void CodeGenData::initStateList( unsigned long length ) +{ + allStates = new RedStateAp[length]; + for ( unsigned long s = 0; s < length; s++ ) + redFsm->stateList.append( allStates+s ); + + /* We get the start state as an offset, set the pointer now. */ + if ( startState >= 0 ) + redFsm->startState = allStates + startState; + if ( errState >= 0 ) + redFsm->errState = allStates + errState; + for ( EntryIdVect::Iter en = entryPointIds; en.lte(); en++ ) + redFsm->entryPoints.insert( allStates + *en ); + + /* The nextStateId is no longer used to assign state ids (they come in set + * from the frontend now), however generation code still depends on it. + * Should eventually remove this variable. */ + redFsm->nextStateId = redFsm->stateList.length(); +} + +void CodeGenData::setStartState( unsigned long startState ) +{ + this->startState = startState; +} + +void CodeGenData::setErrorState( unsigned long errState ) +{ + this->errState = errState; +} + +void CodeGenData::addEntryPoint( char *name, unsigned long entryState ) +{ + entryPointIds.append( entryState ); + entryPointNames.append( name ); +} + +void CodeGenData::initTransList( int snum, unsigned long length ) +{ + /* Could preallocate the out range to save time growing it. For now do + * nothing. */ +} + +void CodeGenData::newTrans( int snum, int tnum, Key lowKey, + Key highKey, long targ, long action ) +{ + /* Get the current state and range. */ + RedStateAp *curState = allStates + snum; + RedTransList &destRange = curState->outRange; + + if ( curState == redFsm->errState ) + return; + + /* Make the new transitions. */ + RedStateAp *targState = targ >= 0 ? (allStates + targ) : + wantComplete ? redFsm->getErrorState() : 0; + RedAction *actionTable = action >= 0 ? (allActionTables + action) : 0; + RedTransAp *trans = redFsm->allocateTrans( targState, actionTable ); + RedTransEl transEl( lowKey, highKey, trans ); + + if ( wantComplete ) { + /* If the machine is to be complete then we need to fill any gaps with + * the error transitions. */ + if ( destRange.length() == 0 ) { + /* Range is currently empty. */ + if ( keyOps->minKey < lowKey ) { + /* The first range doesn't start at the low end. */ + Key fillHighKey = lowKey; + fillHighKey.decrement(); + + /* Create the filler with the state's error transition. */ + RedTransEl newTel( keyOps->minKey, fillHighKey, redFsm->getErrorTrans() ); + destRange.append( newTel ); + } + } + else { + /* The range list is not empty, get the the last range. */ + RedTransEl *last = &destRange[destRange.length()-1]; + Key nextKey = last->highKey; + nextKey.increment(); + if ( nextKey < lowKey ) { + /* There is a gap to fill. Make the high key. */ + Key fillHighKey = lowKey; + fillHighKey.decrement(); + + /* Create the filler with the state's error transtion. */ + RedTransEl newTel( nextKey, fillHighKey, redFsm->getErrorTrans() ); + destRange.append( newTel ); + } + } + } + + /* Filler taken care of. Append the range. */ + destRange.append( RedTransEl( lowKey, highKey, trans ) ); +} + +void CodeGenData::finishTransList( int snum ) +{ + /* Get the current state and range. */ + RedStateAp *curState = allStates + snum; + RedTransList &destRange = curState->outRange; + + if ( curState == redFsm->errState ) + return; + + /* If building a complete machine we may need filler on the end. */ + if ( wantComplete ) { + /* Check if there are any ranges already. */ + if ( destRange.length() == 0 ) { + /* Fill with the whole alphabet. */ + /* Add the range on the lower and upper bound. */ + RedTransEl newTel( keyOps->minKey, keyOps->maxKey, redFsm->getErrorTrans() ); + destRange.append( newTel ); + } + else { + /* Get the last and check for a gap on the end. */ + RedTransEl *last = &destRange[destRange.length()-1]; + if ( last->highKey < keyOps->maxKey ) { + /* Make the high key. */ + Key fillLowKey = last->highKey; + fillLowKey.increment(); + + /* Create the new range with the error trans and append it. */ + RedTransEl newTel( fillLowKey, keyOps->maxKey, redFsm->getErrorTrans() ); + destRange.append( newTel ); + } + } + } +} + +void CodeGenData::setId( int snum, int id ) +{ + RedStateAp *curState = allStates + snum; + curState->id = id; +} + +void CodeGenData::setFinal( int snum ) +{ + RedStateAp *curState = allStates + snum; + curState->isFinal = true; +} + + +void CodeGenData::setStateActions( int snum, long toStateAction, + long fromStateAction, long eofAction ) +{ + RedStateAp *curState = allStates + snum; + if ( toStateAction >= 0 ) + curState->toStateAction = allActionTables + toStateAction; + if ( fromStateAction >= 0 ) + curState->fromStateAction = allActionTables + fromStateAction; + if ( eofAction >= 0 ) + curState->eofAction = allActionTables + eofAction; +} + +void CodeGenData::setEofTrans( int snum, long eofTarget, long actId ) +{ + RedStateAp *curState = allStates + snum; + RedStateAp *targState = allStates + eofTarget; + RedAction *eofAct = allActionTables + actId; + curState->eofTrans = redFsm->allocateTrans( targState, eofAct ); +} + +void CodeGenData::resolveTargetStates( GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: + case GenInlineItem::Next: case GenInlineItem::Entry: + item->targState = allStates + item->targId; + break; + default: + break; + } + + if ( item->children != 0 ) + resolveTargetStates( item->children ); + } +} + +void CodeGenData::closeMachine() +{ + for ( GenActionList::Iter a = actionList; a.lte(); a++ ) + resolveTargetStates( a->inlineList ); + + /* Note that even if we want a complete graph we do not give the error + * state a default transition. All machines break out of the processing + * loop when in the error state. */ + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + for ( GenStateCondList::Iter sci = st->stateCondList; sci.lte(); sci++ ) + st->stateCondVect.append( sci ); + } +} + + +bool CodeGenData::setAlphType( const char *data ) +{ + HostType *alphType = findAlphTypeInternal( data ); + if ( alphType == 0 ) + return false; + + thisKeyOps.setAlphType( alphType ); + return true; +} + +void CodeGenData::initCondSpaceList( ulong length ) +{ + allCondSpaces = new GenCondSpace[length]; + for ( ulong c = 0; c < length; c++ ) + condSpaceList.append( allCondSpaces + c ); +} + +void CodeGenData::newCondSpace( int cnum, int condSpaceId, Key baseKey ) +{ + GenCondSpace *cond = allCondSpaces + cnum; + cond->condSpaceId = condSpaceId; + cond->baseKey = baseKey; +} + +void CodeGenData::condSpaceItem( int cnum, long condActionId ) +{ + GenCondSpace *cond = allCondSpaces + cnum; + cond->condSet.append( allActions + condActionId ); +} + +void CodeGenData::initStateCondList( int snum, ulong length ) +{ + /* Could preallocate these, as we could with transitions. */ +} + +void CodeGenData::addStateCond( int snum, Key lowKey, Key highKey, long condNum ) +{ + RedStateAp *curState = allStates + snum; + + /* Create the new state condition. */ + GenStateCond *stateCond = new GenStateCond; + stateCond->lowKey = lowKey; + stateCond->highKey = highKey; + + /* Assign it a cond space. */ + GenCondSpace *condSpace = allCondSpaces + condNum; + stateCond->condSpace = condSpace; + + curState->stateCondList.append( stateCond ); +} + + +GenCondSpace *CodeGenData::findCondSpace( Key lowKey, Key highKey ) +{ + for ( CondSpaceList::Iter cs = condSpaceList; cs.lte(); cs++ ) { + Key csHighKey = cs->baseKey; + csHighKey += keyOps->alphSize() * (1 << cs->condSet.length()); + + if ( lowKey >= cs->baseKey && highKey <= csHighKey ) + return cs; + } + return 0; +} + +Condition *CodeGenData::findCondition( Key key ) +{ + for ( ConditionList::Iter cond = conditionList; cond.lte(); cond++ ) { + Key upperKey = cond->baseKey + (1 << cond->condSet.length()); + if ( cond->baseKey <= key && key <= upperKey ) + return cond; + } + return 0; +} + +Key CodeGenData::findMaxKey() +{ + Key maxKey = keyOps->maxKey; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + assert( st->outSingle.length() == 0 ); + assert( st->defTrans == 0 ); + + long rangeLen = st->outRange.length(); + if ( rangeLen > 0 ) { + Key highKey = st->outRange[rangeLen-1].highKey; + if ( highKey > maxKey ) + maxKey = highKey; + } + } + return maxKey; +} + +void CodeGenData::findFinalActionRefs() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Rerence count out of single transitions. */ + for ( RedTransList::Iter rtel = st->outSingle; rtel.lte(); rtel++ ) { + if ( rtel->value->action != 0 ) { + rtel->value->action->numTransRefs += 1; + for ( GenActionTable::Iter item = rtel->value->action->key; item.lte(); item++ ) + item->value->numTransRefs += 1; + } + } + + /* Reference count out of range transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + if ( rtel->value->action != 0 ) { + rtel->value->action->numTransRefs += 1; + for ( GenActionTable::Iter item = rtel->value->action->key; item.lte(); item++ ) + item->value->numTransRefs += 1; + } + } + + /* Reference count default transition. */ + if ( st->defTrans != 0 && st->defTrans->action != 0 ) { + st->defTrans->action->numTransRefs += 1; + for ( GenActionTable::Iter item = st->defTrans->action->key; item.lte(); item++ ) + item->value->numTransRefs += 1; + } + + /* Reference count eof transitions. */ + if ( st->eofTrans != 0 && st->eofTrans->action != 0 ) { + st->eofTrans->action->numTransRefs += 1; + for ( GenActionTable::Iter item = st->eofTrans->action->key; item.lte(); item++ ) + item->value->numTransRefs += 1; + } + + /* Reference count to state actions. */ + if ( st->toStateAction != 0 ) { + st->toStateAction->numToStateRefs += 1; + for ( GenActionTable::Iter item = st->toStateAction->key; item.lte(); item++ ) + item->value->numToStateRefs += 1; + } + + /* Reference count from state actions. */ + if ( st->fromStateAction != 0 ) { + st->fromStateAction->numFromStateRefs += 1; + for ( GenActionTable::Iter item = st->fromStateAction->key; item.lte(); item++ ) + item->value->numFromStateRefs += 1; + } + + /* Reference count EOF actions. */ + if ( st->eofAction != 0 ) { + st->eofAction->numEofRefs += 1; + for ( GenActionTable::Iter item = st->eofAction->key; item.lte(); item++ ) + item->value->numEofRefs += 1; + } + } +} + +void CodeGenData::analyzeAction( GenAction *act, GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + /* Only consider actions that are referenced. */ + if ( act->numRefs() > 0 ) { + if ( item->type == GenInlineItem::Goto || item->type == GenInlineItem::GotoExpr ) + redFsm->bAnyActionGotos = true; + else if ( item->type == GenInlineItem::Call || item->type == GenInlineItem::CallExpr ) + redFsm->bAnyActionCalls = true; + else if ( item->type == GenInlineItem::Ret ) + redFsm->bAnyActionRets = true; + + if ( item->type == GenInlineItem::CallExpr || item->type == GenInlineItem::GotoExpr ) + redFsm->bAnyActionByValControl = true; + + } + + /* Check for various things in regular actions. */ + if ( act->numTransRefs > 0 || act->numToStateRefs > 0 || act->numFromStateRefs > 0 ) { + /* Any returns in regular actions? */ + if ( item->type == GenInlineItem::Ret ) + redFsm->bAnyRegActionRets = true; + + /* Any next statements in the regular actions? */ + if ( item->type == GenInlineItem::Next || item->type == GenInlineItem::NextExpr ) + redFsm->bAnyRegNextStmt = true; + + /* Any by value control in regular actions? */ + if ( item->type == GenInlineItem::CallExpr || item->type == GenInlineItem::GotoExpr ) + redFsm->bAnyRegActionByValControl = true; + + /* Any references to the current state in regular actions? */ + if ( item->type == GenInlineItem::Curs ) + redFsm->bAnyRegCurStateRef = true; + + if ( item->type == GenInlineItem::Break ) + redFsm->bAnyRegBreak = true; + } + + if ( item->children != 0 ) + analyzeAction( act, item->children ); + } +} + +void CodeGenData::analyzeActionList( RedAction *redAct, GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + /* Any next statements in the action table? */ + if ( item->type == GenInlineItem::Next || item->type == GenInlineItem::NextExpr ) + redAct->bAnyNextStmt = true; + + /* Any references to the current state. */ + if ( item->type == GenInlineItem::Curs ) + redAct->bAnyCurStateRef = true; + + if ( item->type == GenInlineItem::Break ) + redAct->bAnyBreakStmt = true; + + if ( item->children != 0 ) + analyzeActionList( redAct, item->children ); + } +} + +/* Assign ids to referenced actions. */ +void CodeGenData::assignActionIds() +{ + int nextActionId = 0; + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Only ever interested in referenced actions. */ + if ( act->numRefs() > 0 ) + act->actionId = nextActionId++; + } +} + +void CodeGenData::setValueLimits() +{ + redFsm->maxSingleLen = 0; + redFsm->maxRangeLen = 0; + redFsm->maxKeyOffset = 0; + redFsm->maxIndexOffset = 0; + redFsm->maxActListId = 0; + redFsm->maxActionLoc = 0; + redFsm->maxActArrItem = 0; + redFsm->maxSpan = 0; + redFsm->maxCondSpan = 0; + redFsm->maxFlatIndexOffset = 0; + redFsm->maxCondOffset = 0; + redFsm->maxCondLen = 0; + redFsm->maxCondSpaceId = 0; + redFsm->maxCondIndexOffset = 0; + + /* In both of these cases the 0 index is reserved for no value, so the max + * is one more than it would be if they started at 0. */ + redFsm->maxIndex = redFsm->transSet.length(); + redFsm->maxCond = condSpaceList.length(); + + /* The nextStateId - 1 is the last state id assigned. */ + redFsm->maxState = redFsm->nextStateId - 1; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + if ( csi->condSpaceId > redFsm->maxCondSpaceId ) + redFsm->maxCondSpaceId = csi->condSpaceId; + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Maximum cond length. */ + if ( st->stateCondList.length() > redFsm->maxCondLen ) + redFsm->maxCondLen = st->stateCondList.length(); + + /* Maximum single length. */ + if ( st->outSingle.length() > redFsm->maxSingleLen ) + redFsm->maxSingleLen = st->outSingle.length(); + + /* Maximum range length. */ + if ( st->outRange.length() > redFsm->maxRangeLen ) + redFsm->maxRangeLen = st->outRange.length(); + + /* The key offset index offset for the state after last is not used, skip it.. */ + if ( ! st.last() ) { + redFsm->maxCondOffset += st->stateCondList.length(); + redFsm->maxKeyOffset += st->outSingle.length() + st->outRange.length()*2; + redFsm->maxIndexOffset += st->outSingle.length() + st->outRange.length() + 2; + } + + /* Max cond span. */ + if ( st->condList != 0 ) { + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + if ( span > redFsm->maxCondSpan ) + redFsm->maxCondSpan = span; + } + + /* Max key span. */ + if ( st->transList != 0 ) { + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + if ( span > redFsm->maxSpan ) + redFsm->maxSpan = span; + } + + /* Max cond index offset. */ + if ( ! st.last() ) { + if ( st->condList != 0 ) + redFsm->maxCondIndexOffset += keyOps->span( st->condLowKey, st->condHighKey ); + } + + /* Max flat index offset. */ + if ( ! st.last() ) { + if ( st->transList != 0 ) + redFsm->maxFlatIndexOffset += keyOps->span( st->lowKey, st->highKey ); + redFsm->maxFlatIndexOffset += 1; + } + } + + for ( GenActionTableMap::Iter at = redFsm->actionMap; at.lte(); at++ ) { + /* Maximum id of action lists. */ + if ( at->actListId+1 > redFsm->maxActListId ) + redFsm->maxActListId = at->actListId+1; + + /* Maximum location of items in action array. */ + if ( at->location+1 > redFsm->maxActionLoc ) + redFsm->maxActionLoc = at->location+1; + + /* Maximum values going into the action array. */ + if ( at->key.length() > redFsm->maxActArrItem ) + redFsm->maxActArrItem = at->key.length(); + for ( GenActionTable::Iter item = at->key; item.lte(); item++ ) { + if ( item->value->actionId > redFsm->maxActArrItem ) + redFsm->maxActArrItem = item->value->actionId; + } + } +} + + + +/* Gather various info on the machine. */ +void CodeGenData::analyzeMachine() +{ + /* Find the true count of action references. */ + findFinalActionRefs(); + + /* Check if there are any calls in action code. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Record the occurrence of various kinds of actions. */ + if ( act->numToStateRefs > 0 ) + redFsm->bAnyToStateActions = true; + if ( act->numFromStateRefs > 0 ) + redFsm->bAnyFromStateActions = true; + if ( act->numEofRefs > 0 ) + redFsm->bAnyEofActions = true; + if ( act->numTransRefs > 0 ) + redFsm->bAnyRegActions = true; + + /* Recurse through the action's parse tree looking for various things. */ + analyzeAction( act, act->inlineList ); + } + + /* Analyze reduced action lists. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + for ( GenActionTable::Iter act = redAct->key; act.lte(); act++ ) + analyzeActionList( redAct, act->value->inlineList ); + } + + /* Find states that have transitions with actions that have next + * statements. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Check any actions out of outSinge. */ + for ( RedTransList::Iter rtel = st->outSingle; rtel.lte(); rtel++ ) { + if ( rtel->value->action != 0 && rtel->value->action->anyCurStateRef() ) + st->bAnyRegCurStateRef = true; + } + + /* Check any actions out of outRange. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + if ( rtel->value->action != 0 && rtel->value->action->anyCurStateRef() ) + st->bAnyRegCurStateRef = true; + } + + /* Check any action out of default. */ + if ( st->defTrans != 0 && st->defTrans->action != 0 && + st->defTrans->action->anyCurStateRef() ) + st->bAnyRegCurStateRef = true; + + if ( st->stateCondList.length() > 0 ) + redFsm->bAnyConditions = true; + + if ( st->eofTrans != 0 ) + redFsm->bAnyEofTrans = true; + } + + /* Assign ids to actions that are referenced. */ + assignActionIds(); + + /* Set the maximums of various values used for deciding types. */ + setValueLimits(); +} + +void CodeGenData::write_option_error( InputLoc &loc, char *arg ) +{ + source_warning(loc) << "unrecognized write option \"" << arg << "\"" << endl; +} + +/* returns true if the following section should generate line directives. */ +bool CodeGenData::writeStatement( InputLoc &loc, int nargs, char **args ) +{ + bool followLineDirective = false; + + if ( strcmp( args[0], "data" ) == 0 ) { + out << '\n'; + genLineDirective( out ); + followLineDirective = true; + + for ( int i = 1; i < nargs; i++ ) { + if ( strcmp( args[i], "noerror" ) == 0 ) + noError = true; + else if ( strcmp( args[i], "noprefix" ) == 0 ) + noPrefix = true; + else if ( strcmp( args[i], "nofinal" ) == 0 ) + noFinal = true; + else if ( strcmp( args[i], "noentry" ) == 0 ) + noEntry = true; + else + write_option_error( loc, args[i] ); + } + writeData(); + } + else if ( strcmp( args[0], "init" ) == 0 ) { + out << '\n'; + genLineDirective( out ); + followLineDirective = true; + + for ( int i = 1; i < nargs; i++ ) { + if ( strcmp( args[i], "nocs" ) == 0 ) + noCS = true; + else + write_option_error( loc, args[i] ); + } + writeInit(); + } + else if ( strcmp( args[0], "exec" ) == 0 ) { + out << '\n'; + genLineDirective( out ); + followLineDirective = true; + + for ( int i = 1; i < nargs; i++ ) { + if ( strcmp( args[i], "noend" ) == 0 ) + noEnd = true; + else + write_option_error( loc, args[i] ); + } + writeExec(); + } + else if ( strcmp( args[0], "exports" ) == 0 ) { + out << '\n'; + genLineDirective( out ); + followLineDirective = true; + + for ( int i = 1; i < nargs; i++ ) + write_option_error( loc, args[i] ); + writeExports(); + } + else if ( strcmp( args[0], "start" ) == 0 ) { + for ( int i = 1; i < nargs; i++ ) + write_option_error( loc, args[i] ); + writeStart(); + } + else if ( strcmp( args[0], "first_final" ) == 0 ) { + for ( int i = 1; i < nargs; i++ ) + write_option_error( loc, args[i] ); + writeFirstFinal(); + } + else if ( strcmp( args[0], "error" ) == 0 ) { + for ( int i = 1; i < nargs; i++ ) + write_option_error( loc, args[i] ); + writeError(); + } + else { + /* EMIT An error here. */ + source_error(loc) << "unrecognized write command \"" << + args[0] << "\"" << endl; + } + return followLineDirective; +} + +ostream &CodeGenData::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &CodeGenData::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + + diff --git a/contrib/tools/ragel6/gendata.h b/contrib/tools/ragel6/gendata.h new file mode 100644 index 0000000000..6c3d3852a1 --- /dev/null +++ b/contrib/tools/ragel6/gendata.h @@ -0,0 +1,189 @@ +/* + * Copyright 2005-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GENDATA_H +#define _GENDATA_H + +#include <iostream> +#include "config.h" +#include "redfsm.h" +#include "common.h" + +using std::ostream; + +extern bool generateDot; + +struct NameInst; +typedef DList<GenAction> GenActionList; + +typedef unsigned long ulong; + +extern int gblErrorCount; + +struct CodeGenData; + +typedef AvlMap<char *, CodeGenData*, CmpStr> CodeGenMap; +typedef AvlMapEl<char *, CodeGenData*> CodeGenMapEl; + +void cdLineDirective( ostream &out, const char *fileName, int line ); +void javaLineDirective( ostream &out, const char *fileName, int line ); +void goLineDirective( ostream &out, const char *fileName, int line ); +void rubyLineDirective( ostream &out, const char *fileName, int line ); +void csharpLineDirective( ostream &out, const char *fileName, int line ); +void ocamlLineDirective( ostream &out, const char *fileName, int line ); +void genLineDirective( ostream &out ); +void lineDirective( ostream &out, const char *fileName, int line ); + +string itoa( int i ); + +/*********************************/ + +struct CodeGenData +{ + /* + * The interface to the code generator. + */ + virtual void finishRagelDef() {} + + /* These are invoked by the corresponding write statements. */ + virtual void writeData() {}; + virtual void writeInit() {}; + virtual void writeExec() {}; + virtual void writeExports() {}; + virtual void writeStart() {}; + virtual void writeFirstFinal() {}; + virtual void writeError() {}; + + /* This can also be overwridden to modify the processing of write + * statements. */ + virtual bool writeStatement( InputLoc &loc, int nargs, char **args ); + + /********************/ + + CodeGenData( ostream &out ); + virtual ~CodeGenData() {} + + /* + * Collecting the machine. + */ + + const char *sourceFileName; + const char *fsmName; + ostream &out; + RedFsmAp *redFsm; + GenAction *allActions; + RedAction *allActionTables; + Condition *allConditions; + GenCondSpace *allCondSpaces; + RedStateAp *allStates; + NameInst **nameIndex; + int startState; + int errState; + GenActionList actionList; + ConditionList conditionList; + CondSpaceList condSpaceList; + GenInlineList *getKeyExpr; + GenInlineList *accessExpr; + GenInlineList *prePushExpr; + GenInlineList *postPopExpr; + + /* Overriding variables. */ + GenInlineList *pExpr; + GenInlineList *peExpr; + GenInlineList *eofExpr; + GenInlineList *csExpr; + GenInlineList *topExpr; + GenInlineList *stackExpr; + GenInlineList *actExpr; + GenInlineList *tokstartExpr; + GenInlineList *tokendExpr; + GenInlineList *dataExpr; + + KeyOps thisKeyOps; + bool wantComplete; + EntryIdVect entryPointIds; + EntryNameVect entryPointNames; + bool hasLongestMatch; + ExportList exportList; + + /* Write options. */ + bool noEnd; + bool noPrefix; + bool noFinal; + bool noError; + bool noEntry; + bool noCS; + + void createMachine(); + void initActionList( unsigned long length ); + void newAction( int anum, const char *name, const InputLoc &loc, GenInlineList *inlineList ); + void initActionTableList( unsigned long length ); + void initStateList( unsigned long length ); + void setStartState( unsigned long startState ); + void setErrorState( unsigned long errState ); + void addEntryPoint( char *name, unsigned long entryState ); + void setId( int snum, int id ); + void setFinal( int snum ); + void initTransList( int snum, unsigned long length ); + void newTrans( int snum, int tnum, Key lowKey, Key highKey, + long targ, long act ); + void finishTransList( int snum ); + void setStateActions( int snum, long toStateAction, + long fromStateAction, long eofAction ); + void setEofTrans( int snum, long targ, long eofAction ); + void setForcedErrorState() + { redFsm->forcedErrorState = true; } + + + void initCondSpaceList( ulong length ); + void condSpaceItem( int cnum, long condActionId ); + void newCondSpace( int cnum, int condSpaceId, Key baseKey ); + + void initStateCondList( int snum, ulong length ); + void addStateCond( int snum, Key lowKey, Key highKey, long condNum ); + + GenCondSpace *findCondSpace( Key lowKey, Key highKey ); + Condition *findCondition( Key key ); + + bool setAlphType( const char *data ); + + void resolveTargetStates( GenInlineList *inlineList ); + Key findMaxKey(); + + /* Gather various info on the machine. */ + void analyzeActionList( RedAction *redAct, GenInlineList *inlineList ); + void analyzeAction( GenAction *act, GenInlineList *inlineList ); + void findFinalActionRefs(); + void analyzeMachine(); + + void closeMachine(); + void setValueLimits(); + void assignActionIds(); + + ostream &source_warning( const InputLoc &loc ); + ostream &source_error( const InputLoc &loc ); + void write_option_error( InputLoc &loc, char *arg ); +}; + +CodeGenData *makeCodeGen( const char *sourceFileName, + const char *fsmName, ostream &out ); + +#endif diff --git a/contrib/tools/ragel6/gocodegen.cpp b/contrib/tools/ragel6/gocodegen.cpp new file mode 100644 index 0000000000..d28f0f098a --- /dev/null +++ b/contrib/tools/ragel6/gocodegen.cpp @@ -0,0 +1,775 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "gocodegen.h" +#include "ragel.h" +#include "redfsm.h" +#include "gendata.h" +#include <sstream> +#include <string> +#include <assert.h> + + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + +/* + * Go Specific + */ + +void goLineDirective( ostream &out, const char *fileName, int line ) +{ + out << "//line " << fileName << ":" << line << endl; +} + +void GoCodeGen::genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + goLineDirective( out, filter->fileName, filter->line + 1 ); +} + +unsigned int GoCodeGen::arrayTypeSize( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + return arrayType->size; +} + +string GoCodeGen::ARRAY_TYPE( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + + string ret = arrayType->data1; + if ( arrayType->data2 != 0 ) { + ret += " "; + ret += arrayType->data2; + } + return ret; +} + + +/* Write out the fsm name. */ +string GoCodeGen::FSM_NAME() +{ + return fsmName; +} + +/* Emit the offset of the start state as a decimal integer. */ +string GoCodeGen::START_STATE_ID() +{ + ostringstream ret; + ret << redFsm->startState->id; + return ret.str(); +}; + +/* Write out the array of actions. */ +std::ostream &GoCodeGen::ACTIONS_ARRAY() +{ + out << " 0, "; + int totalActions = 1; + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + /* Write out the length, which will never be the last character. */ + out << act->key.length() << ", "; + if ( totalActions++ % IALL == 0 ) + out << endl << " "; + + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) { + out << item->value->actionId << ", "; + if ( ! (act.last() && item.last()) ) { + if ( totalActions++ % IALL == 0 ) + out << endl << " "; + } + } + } + out << endl; + return out; +} + + +string GoCodeGen::ACCESS() +{ + ostringstream ret; + if ( accessExpr != 0 ) + INLINE_LIST( ret, accessExpr, 0, false, false ); + return ret.str(); +} + + +string GoCodeGen::P() +{ + ostringstream ret; + if ( pExpr == 0 ) + ret << "p"; + else { + ret << "("; + INLINE_LIST( ret, pExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string GoCodeGen::PE() +{ + ostringstream ret; + if ( peExpr == 0 ) + ret << "pe"; + else { + ret << "("; + INLINE_LIST( ret, peExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string GoCodeGen::vEOF() +{ + ostringstream ret; + if ( eofExpr == 0 ) + ret << "eof"; + else { + ret << "("; + INLINE_LIST( ret, eofExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string GoCodeGen::vCS() +{ + ostringstream ret; + if ( csExpr == 0 ) + ret << ACCESS() << "cs"; + else { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, csExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string GoCodeGen::TOP() +{ + ostringstream ret; + if ( topExpr == 0 ) + ret << ACCESS() + "top"; + else { + ret << "("; + INLINE_LIST( ret, topExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string GoCodeGen::STACK() +{ + ostringstream ret; + if ( stackExpr == 0 ) + ret << ACCESS() + "stack"; + else { + ret << "("; + INLINE_LIST( ret, stackExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string GoCodeGen::ACT() +{ + ostringstream ret; + if ( actExpr == 0 ) + ret << ACCESS() + "act"; + else { + ret << "("; + INLINE_LIST( ret, actExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string GoCodeGen::TOKSTART() +{ + ostringstream ret; + if ( tokstartExpr == 0 ) + ret << ACCESS() + "ts"; + else { + ret << "("; + INLINE_LIST( ret, tokstartExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string GoCodeGen::TOKEND() +{ + ostringstream ret; + if ( tokendExpr == 0 ) + ret << ACCESS() + "te"; + else { + ret << "("; + INLINE_LIST( ret, tokendExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string GoCodeGen::GET_WIDE_KEY() +{ + if ( redFsm->anyConditions() ) + return "_widec"; + else + return GET_KEY(); +} + +string GoCodeGen::GET_WIDE_KEY( RedStateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) + return "_widec"; + else + return GET_KEY(); +} + +string GoCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + ret << DATA() << "[" << P() << "]"; + } + return ret.str(); +} + +/* Write out level number of tabs. Makes the nested binary search nice + * looking. */ +string GoCodeGen::TABS( int level ) +{ + string result; + while ( level-- > 0 ) + result += "\t"; + return result; +} + +/* Write out a key from the fsm code gen. Depends on wether or not the key is + * signed. */ +string GoCodeGen::KEY( Key key ) +{ + ostringstream ret; + if ( keyOps->isSigned || !hostLang->explicitUnsigned ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal() << 'u'; + return ret.str(); +} + +bool GoCodeGen::isAlphTypeSigned() +{ + return keyOps->isSigned; +} + +bool GoCodeGen::isWideAlphTypeSigned() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + return isAlphTypeSigned(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + return wideType->isSigned; + } +} + +string GoCodeGen::WIDE_KEY( RedStateAp *state, Key key ) +{ + if ( state->stateCondList.length() > 0 ) { + ostringstream ret; + if ( isWideAlphTypeSigned() ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal() << 'u'; + return ret.str(); + } + else { + return KEY( key ); + } +} + + + +void GoCodeGen::EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ) +{ + /* The parser gives fexec two children. The double brackets are for D + * code. If the inline list is a single word it will get interpreted as a + * C-style cast by the D compiler. */ + ret << P() << " = ("; + INLINE_LIST( ret, item->children, targState, inFinish, false ); + ret << ") - 1" << endl; +} + +void GoCodeGen::LM_SWITCH( ostream &ret, GenInlineItem *item, + int targState, int inFinish, bool csForced ) +{ + ret << + " switch " << ACT() << " {" << endl; + + for ( GenInlineList::Iter lma = *item->children; lma.lte(); lma++ ) { + /* Write the case label, the action and the case break. */ + if ( lma->lmId < 0 ) { + ret << " default:" << endl; + } + else + ret << " case " << lma->lmId << ":" << endl; + + /* Write the block and close it off. */ + ret << " {"; + INLINE_LIST( ret, lma->children, targState, inFinish, csForced ); + ret << "}" << endl; + } + + ret << + " }" << endl << + " "; +} + +void GoCodeGen::SET_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = " << item->lmId << ";"; +} + +void GoCodeGen::SET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + /* The tokend action sets tokend. */ + ret << TOKEND() << " = " << P(); + if ( item->offset != 0 ) + out << "+" << item->offset; + out << endl; +} + +void GoCodeGen::GET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + ret << TOKEND(); +} + +void GoCodeGen::INIT_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << NULL_ITEM() << endl; +} + +void GoCodeGen::INIT_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = 0" << endl; +} + +void GoCodeGen::SET_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << P() << endl; +} + +void GoCodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish, bool csForced ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << "{"; + INLINE_LIST( ret, item->children, targState, inFinish, csForced ); + ret << "}"; + } +} + + +/* Write out an inline tree structure. Walks the list and possibly calls out + * to virtual functions than handle language specific items in the tree. */ +void GoCodeGen::INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish, bool csForced ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Text: + ret << item->data; + break; + case GenInlineItem::Goto: + GOTO( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Call: + CALL( ret, item->targState->id, targState, inFinish ); + break; + case GenInlineItem::Next: + NEXT( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Ret: + RET( ret, inFinish ); + break; + case GenInlineItem::PChar: + ret << P(); + break; + case GenInlineItem::Char: + ret << GET_KEY(); + break; + case GenInlineItem::Hold: + ret << P() << "--" << endl; + break; + case GenInlineItem::Exec: + EXEC( ret, item, targState, inFinish ); + break; + case GenInlineItem::Curs: + CURS( ret, inFinish ); + break; + case GenInlineItem::Targs: + TARGS( ret, inFinish, targState ); + break; + case GenInlineItem::Entry: + ret << item->targState->id; + break; + case GenInlineItem::GotoExpr: + GOTO_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::CallExpr: + CALL_EXPR( ret, item, targState, inFinish ); + break; + case GenInlineItem::NextExpr: + NEXT_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::LmSwitch: + LM_SWITCH( ret, item, targState, inFinish, csForced ); + break; + case GenInlineItem::LmSetActId: + SET_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokEnd: + SET_TOKEND( ret, item ); + break; + case GenInlineItem::LmGetTokEnd: + GET_TOKEND( ret, item ); + break; + case GenInlineItem::LmInitTokStart: + INIT_TOKSTART( ret, item ); + break; + case GenInlineItem::LmInitAct: + INIT_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokStart: + SET_TOKSTART( ret, item ); + break; + case GenInlineItem::SubAction: + SUB_ACTION( ret, item, targState, inFinish, csForced ); + break; + case GenInlineItem::Break: + BREAK( ret, targState, csForced ); + break; + } + } +} +/* Write out paths in line directives. Escapes any special characters. */ +string GoCodeGen::LDIR_PATH( char *path ) +{ + ostringstream ret; + for ( char *pc = path; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + ret << "\\\\"; + else + ret << *pc; + } + return ret.str(); +} + +void GoCodeGen::ACTION( ostream &ret, GenAction *action, int targState, + bool inFinish, bool csForced ) +{ + /* Write the preprocessor line info for going into the source file. */ + goLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + INLINE_LIST( ret, action->inlineList, targState, inFinish, csForced ); + ret << endl; +} + +void GoCodeGen::CONDITION( ostream &ret, GenAction *condition ) +{ + INLINE_LIST( ret, condition->inlineList, 0, false, false ); +} + +string GoCodeGen::ERROR_STATE() +{ + ostringstream ret; + if ( redFsm->errState != 0 ) + ret << redFsm->errState->id; + else + ret << "-1"; + return ret.str(); +} + +string GoCodeGen::FIRST_FINAL_STATE() +{ + ostringstream ret; + if ( redFsm->firstFinState != 0 ) + ret << redFsm->firstFinState->id; + else + ret << redFsm->nextStateId; + return ret.str(); +} + +void GoCodeGen::writeInit() +{ + out << " {" << endl; + + if ( !noCS ) + out << " " << vCS() << " = " << START() << endl; + + /* If there are any calls, then the stack top needs initialization. */ + if ( redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << " " << TOP() << " = 0" << endl; + + if ( hasLongestMatch ) { + out << + " " << TOKSTART() << " = " << NULL_ITEM() << endl << + " " << TOKEND() << " = " << NULL_ITEM() << endl << + " " << ACT() << " = 0" << endl; + } + out << " }" << endl; +} + +string GoCodeGen::DATA() +{ + ostringstream ret; + if ( dataExpr == 0 ) + ret << ACCESS() + "data"; + else { + ret << "("; + INLINE_LIST( ret, dataExpr, 0, false, false ); + ret << ")"; + } + return ret.str(); +} + +string GoCodeGen::DATA_PREFIX() +{ + if ( !noPrefix ) + return FSM_NAME() + "_"; + return ""; +} + +/* Emit the alphabet data type. */ +string GoCodeGen::ALPH_TYPE() +{ + string ret = keyOps->alphType->data1; + if ( keyOps->alphType->data2 != 0 ) { + ret += " "; + ret += + keyOps->alphType->data2; + } + return ret; +} + +/* Emit the alphabet data type. */ +string GoCodeGen::WIDE_ALPH_TYPE() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + ret = ALPH_TYPE(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + assert( wideType != 0 ); + + ret = wideType->data1; + if ( wideType->data2 != 0 ) { + ret += " "; + ret += wideType->data2; + } + } + return ret; +} + +void GoCodeGen::STATE_IDS() +{ + if ( redFsm->startState != 0 ) + CONST( "int", START() ) << " = " << START_STATE_ID() << endl; + + if ( !noFinal ) + CONST( "int" , FIRST_FINAL() ) << " = " << FIRST_FINAL_STATE() << endl; + + if ( !noError ) + CONST( "int", ERROR() ) << " = " << ERROR_STATE() << endl; + + out << endl; + + if ( !noEntry && entryPointNames.length() > 0 ) { + for ( EntryNameVect::Iter en = entryPointNames; en.lte(); en++ ) { + CONST( "int", DATA_PREFIX() + "en_" + *en ) << + " = " << entryPointIds[en.pos()] << endl; + } + out << endl; + } +} + +void GoCodeGen::writeStart() +{ + out << START_STATE_ID(); +} + +void GoCodeGen::writeFirstFinal() +{ + out << FIRST_FINAL_STATE(); +} + +void GoCodeGen::writeError() +{ + out << ERROR_STATE(); +} + +void GoCodeGen::finishRagelDef() +{ + if ( codeStyle == GenGoto || codeStyle == GenFGoto || + codeStyle == GenIpGoto || codeStyle == GenSplit ) + { + /* For directly executable machines there is no required state + * ordering. Choose a depth-first ordering to increase the + * potential for fall-throughs. */ + redFsm->depthFirstOrdering(); + } + else { + /* The frontend will do this for us, but it may be a good idea to + * force it if the intermediate file is edited. */ + redFsm->sortByStateId(); + } + + /* Choose default transitions and the single transition. */ + redFsm->chooseDefaultSpan(); + + /* Maybe do flat expand, otherwise choose single. */ + if ( codeStyle == GenFlat || codeStyle == GenFFlat ) + redFsm->makeFlat(); + else + redFsm->chooseSingle(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + if ( codeStyle == GenSplit ) + redFsm->partitionFsm( numSplitPartitions ); + + if ( codeStyle == GenIpGoto || codeStyle == GenSplit ) + redFsm->setInTrans(); + + /* Anlayze Machine will find the final action reference counts, among + * other things. We will use these in reporting the usage + * of fsm directives in action code. */ + analyzeMachine(); + + /* Determine if we should use indicies. */ + calcIndexSize(); +} + +ostream &GoCodeGen::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &GoCodeGen::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + + +/* + * Go implementation. + * + */ + +std::ostream &GoCodeGen::OPEN_ARRAY( string type, string name ) +{ + out << "var " << name << " []" << type << " = []" << type << "{" << endl; + return out; +} + +std::ostream &GoCodeGen::CLOSE_ARRAY() +{ + return out << "}" << endl; +} + +std::ostream &GoCodeGen::STATIC_VAR( string type, string name ) +{ + out << "var " << name << " " << type; + return out; +} + +std::ostream &GoCodeGen::CONST( string type, string name ) +{ + out << "const " << name << " " << type; + return out; +} + +string GoCodeGen::UINT( ) +{ + return "uint"; +} + +string GoCodeGen::INT() +{ + return "int"; +} + +string GoCodeGen::CAST( string type, string expr ) +{ + return type + "(" + expr + ")"; +} + +string GoCodeGen::NULL_ITEM() +{ + return "0"; +} + +void GoCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + out << "const " << DATA_PREFIX() << "ex_" << ex->name << " = " << + KEY(ex->key) << endl; + } + out << endl; + } +} diff --git a/contrib/tools/ragel6/gocodegen.h b/contrib/tools/ragel6/gocodegen.h new file mode 100644 index 0000000000..50e8804b55 --- /dev/null +++ b/contrib/tools/ragel6/gocodegen.h @@ -0,0 +1,181 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOCODEGEN_H +#define _GOCODEGEN_H + +#include <iostream> +#include <string> +#include <stdio.h> +#include "common.h" +#include "gendata.h" + +using std::string; +using std::ostream; + +/* Integer array line length. */ +#define IALL 8 + +/* Forwards. */ +struct RedFsmAp; +struct RedStateAp; +struct CodeGenData; +struct GenAction; +struct NameInst; +struct GenInlineItem; +struct GenInlineList; +struct RedAction; +struct LongestMatch; +struct LongestMatchPart; + +class GoCodeGen : public CodeGenData +{ +public: + GoCodeGen( ostream &out ) + : CodeGenData(out) {} + + virtual ~GoCodeGen() {} + + virtual void finishRagelDef(); + virtual void writeInit(); + virtual void writeStart(); + virtual void writeFirstFinal(); + virtual void writeError(); + virtual void writeExports(); +protected: + string FSM_NAME(); + string START_STATE_ID(); + ostream &ACTIONS_ARRAY(); + string GET_WIDE_KEY(); + string GET_WIDE_KEY( RedStateAp *state ); + string TABS( int level ); + string KEY( Key key ); + string WIDE_KEY( RedStateAp *state, Key key ); + string LDIR_PATH( char *path ); + virtual void ACTION( ostream &ret, GenAction *action, int targState, + bool inFinish, bool csForced ); + void CONDITION( ostream &ret, GenAction *condition ); + string ALPH_TYPE(); + string WIDE_ALPH_TYPE(); + string ARRAY_TYPE( unsigned long maxVal ); + + bool isAlphTypeSigned(); + bool isWideAlphTypeSigned(); + + virtual string CAST( string type, string expr ); + virtual string UINT(); + virtual string INT(); + virtual string NULL_ITEM(); + virtual string GET_KEY(); + + string P(); + string PE(); + string vEOF(); + + string ACCESS(); + string vCS(); + string STACK(); + string TOP(); + string TOKSTART(); + string TOKEND(); + string ACT(); + string DATA(); + + string DATA_PREFIX(); + string PM() { return "_" + DATA_PREFIX() + "partition_map"; } + string C() { return "_" + DATA_PREFIX() + "cond_spaces"; } + string CK() { return "_" + DATA_PREFIX() + "cond_keys"; } + string K() { return "_" + DATA_PREFIX() + "trans_keys"; } + string I() { return "_" + DATA_PREFIX() + "indicies"; } + string CO() { return "_" + DATA_PREFIX() + "cond_offsets"; } + string KO() { return "_" + DATA_PREFIX() + "key_offsets"; } + string IO() { return "_" + DATA_PREFIX() + "index_offsets"; } + string CL() { return "_" + DATA_PREFIX() + "cond_lengths"; } + string SL() { return "_" + DATA_PREFIX() + "single_lengths"; } + string RL() { return "_" + DATA_PREFIX() + "range_lengths"; } + string A() { return "_" + DATA_PREFIX() + "actions"; } + string TA() { return "_" + DATA_PREFIX() + "trans_actions"; } + string TT() { return "_" + DATA_PREFIX() + "trans_targs"; } + string TSA() { return "_" + DATA_PREFIX() + "to_state_actions"; } + string FSA() { return "_" + DATA_PREFIX() + "from_state_actions"; } + string EA() { return "_" + DATA_PREFIX() + "eof_actions"; } + string ET() { return "_" + DATA_PREFIX() + "eof_trans"; } + string SP() { return "_" + DATA_PREFIX() + "key_spans"; } + string CSP() { return "_" + DATA_PREFIX() + "cond_key_spans"; } + string START() { return DATA_PREFIX() + "start"; } + string ERROR() { return DATA_PREFIX() + "error"; } + string FIRST_FINAL() { return DATA_PREFIX() + "first_final"; } + string CTXDATA() { return DATA_PREFIX() + "ctxdata"; } + + void INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish, bool csForced ); + virtual void GOTO( ostream &ret, int gotoDest, bool inFinish ) = 0; + virtual void CALL( ostream &ret, int callDest, int targState, bool inFinish ) = 0; + virtual void NEXT( ostream &ret, int nextDest, bool inFinish ) = 0; + virtual void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, + int targState, bool inFinish ) = 0; + virtual void RET( ostream &ret, bool inFinish ) = 0; + virtual void BREAK( ostream &ret, int targState, bool csForced ) = 0; + virtual void CURS( ostream &ret, bool inFinish ) = 0; + virtual void TARGS( ostream &ret, bool inFinish, int targState ) = 0; + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, + int inFinish, bool csForced ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + virtual void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish, bool csForced ); + void STATE_IDS(); + + string ERROR_STATE(); + string FIRST_FINAL_STATE(); + + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + virtual ostream &CONST( string type, string name ); + + ostream &source_warning(const InputLoc &loc); + ostream &source_error(const InputLoc &loc); + + unsigned int arrayTypeSize( unsigned long maxVal ); + + bool outLabelUsed; + bool testEofUsed; + bool againLabelUsed; + bool useIndicies; + + void genLineDirective( ostream &out ); + +public: + /* Determine if we should use indicies. */ + virtual void calcIndexSize() {} +}; + +#endif diff --git a/contrib/tools/ragel6/gofflat.cpp b/contrib/tools/ragel6/gofflat.cpp new file mode 100644 index 0000000000..59e729c76b --- /dev/null +++ b/contrib/tools/ragel6/gofflat.cpp @@ -0,0 +1,380 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "gofflat.h" +#include "redfsm.h" +#include "gendata.h" + +using std::endl; + +std::ostream &GoFFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &GoFFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &GoFFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + +/* Write out the function for a transition. */ +std::ostream &GoFFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &GoFFlatCodeGen::TO_STATE_ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &GoFFlatCodeGen::FROM_STATE_ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GoFFlatCodeGen::EOF_ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true, false ); + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &GoFFlatCodeGen::ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +void GoFFlatCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + endl; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + endl; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + endl; + } + + STATE_IDS(); +} + +void GoFFlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {" << endl << + " var _slen " << INT() << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " var _ps " << INT() << endl; + + out << " var _trans " << INT() << endl; + + if ( redFsm->anyConditions() ) + out << " var _cond " << INT() << endl; + + out << + " var _keys " << INT() << endl << + " var _inds " << INT() << endl; + + if ( redFsm->anyConditions() ) { + out << + " var _conds " << INT() << endl << + " var _widec " << WIDE_ALPH_TYPE() << endl; + } + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " == " << PE() << " {" << endl << + " goto _test_eof" << endl << + " }" << endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + out << "_resume:" << endl; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch " << FSA() << "[" << vCS() << "] {" << endl; + FROM_STATE_ACTION_SWITCH(1); + out << + " }" << endl << + endl; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:" << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << endl; + + out << + " " << vCS() << " = " << CAST(INT(), TT() + "[_trans]") << endl << + endl; + + if ( redFsm->anyRegActions() ) { + out << + " if " << TA() << "[_trans] == 0 {" << endl << + " goto _again" << endl << + " }" << endl << + endl << + " switch " << TA() << "[_trans] {" << endl; + ACTION_SWITCH(1); + out << + " }" << endl << + endl; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:" << endl; + + if ( redFsm->anyToStateActions() ) { + out << + " switch " << TSA() << "[" << vCS() << "] {" << endl; + TO_STATE_ACTION_SWITCH(1); + out << + " }" << endl << + endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + if ( !noEnd ) { + out << + " if " << P() << "++; " << P() << " != " << PE() << " {" + " goto _resume" << endl << + " }" << endl; + } + else { + out << + " " << P() << "++" << endl << + " goto _resume" << endl; + } + + if ( testEofUsed ) + out << " _test_eof: {}" << endl; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << " {" << endl; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0 {" << endl << + " _trans = " << CAST(INT(), ET() + "[" + vCS() + "] - 1") << endl << + " goto _eof_trans" << endl << + " }" << endl; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch " << EA() << "[" << vCS() << "] {" << endl; + EOF_ACTION_SWITCH(2); + out << + " }" << endl; + } + + out << + " }" << endl << + endl; + } + + if ( outLabelUsed ) + out << " _out: {}" << endl; + + out << " }" << endl; +} diff --git a/contrib/tools/ragel6/gofflat.h b/contrib/tools/ragel6/gofflat.h new file mode 100644 index 0000000000..bd3bde4cbf --- /dev/null +++ b/contrib/tools/ragel6/gofflat.h @@ -0,0 +1,58 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOFFLAT_H +#define _GOFFLAT_H + +#include <iostream> +#include "goflat.h" + +/* Forwards. */ +struct CodeGenData; + +/* + * FFlatCodeGen + */ +class GoFFlatCodeGen + : public GoFlatCodeGen +{ +public: + GoFFlatCodeGen( ostream &out ) + : GoFlatCodeGen(out) {} + +protected: + std::ostream &TO_STATE_ACTION_SWITCH( int level ); + std::ostream &FROM_STATE_ACTION_SWITCH( int level ); + std::ostream &EOF_ACTION_SWITCH( int level ); + std::ostream &ACTION_SWITCH( int level ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/contrib/tools/ragel6/gofgoto.cpp b/contrib/tools/ragel6/gofgoto.cpp new file mode 100644 index 0000000000..4f6ce36883 --- /dev/null +++ b/contrib/tools/ragel6/gofgoto.cpp @@ -0,0 +1,297 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "gofgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +using std::endl; + +std::ostream &GoFGotoCodeGen::EXEC_ACTIONS() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* We are at the start of a glob, write the case. */ + out << "f" << redAct->actListId << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << TABS(1) << "goto _again" << endl; + } + } + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &GoFGotoCodeGen::TO_STATE_ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &GoFGotoCodeGen::FROM_STATE_ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GoFGotoCodeGen::EOF_ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true, false ); + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &GoFGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << TABS(2) << "case " << st->id << ":" << endl; + + /* Jump to the func. */ + out << TABS(3) << "goto f" << st->eofAction->actListId << endl; + } + } + + return out; +} + +unsigned int GoFGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + return act; +} + +unsigned int GoFGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + return act; +} + +unsigned int GoFGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + return act; +} + +void GoFGotoCodeGen::writeData() +{ + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + STATE_IDS(); +} + +void GoFGotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " {" << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " var _ps " << INT() << " = 0" << endl; + + if ( redFsm->anyConditions() ) + out << " var _widec " << WIDE_ALPH_TYPE() << endl; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " == " << PE() << " {" << endl << + " goto _test_eof" << endl << + " }" << endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + out << "_resume:" << endl; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch " << FSA() << "[" << vCS() << "] {" << endl; + FROM_STATE_ACTION_SWITCH(1); + out << + " }" << endl << + endl; + } + + out << + " switch " << vCS() << " {" << endl; + STATE_GOTOS(1); + out << + " }" << endl << + endl; + TRANSITIONS() << + endl; + + if ( redFsm->anyRegActions() ) + EXEC_ACTIONS() << endl; + + out << "_again:" << endl; + + if ( redFsm->anyToStateActions() ) { + out << + " switch " << TSA() << "[" << vCS() << "] {" << endl; + TO_STATE_ACTION_SWITCH(1); + out << + " }" << endl << + endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + if ( !noEnd ) { + out << + " if " << P() << "++; " << P() << " != " << PE() << " {" << endl << + " goto _resume" << endl << + " }" << endl; + } + else { + out << + " " << P() << "++" << endl << + " goto _resume" << endl; + } + + if ( testEofUsed ) + out << " _test_eof: {}" << endl; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << " {" << endl; + + if ( redFsm->anyEofTrans() ) { + out << + " switch " << vCS() << " {" << endl; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << + " case " << st->id << ":" << endl << + " goto tr" << st->eofTrans->id << endl; + } + + out << + " }" << endl; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch " << EA() << "[" << vCS() << "] {" << endl; + EOF_ACTION_SWITCH(2); + out << + " }" << endl; + } + + out << + " }" << endl << + endl; + } + + if ( outLabelUsed ) + out << " _out: {}" << endl; + + out << " }" << endl; +} diff --git a/contrib/tools/ragel6/gofgoto.h b/contrib/tools/ragel6/gofgoto.h new file mode 100644 index 0000000000..c53e450436 --- /dev/null +++ b/contrib/tools/ragel6/gofgoto.h @@ -0,0 +1,54 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOFGOTO_H +#define _GOFGOTO_H + +#include <iostream> +#include "gogoto.h" + +/* Forwards. */ +struct CodeGenData; + + +class GoFGotoCodeGen + : public GoGotoCodeGen +{ +public: + GoFGotoCodeGen( ostream &out ) + : GoGotoCodeGen(out) {} + + std::ostream &EXEC_ACTIONS(); + std::ostream &TO_STATE_ACTION_SWITCH( int level ); + std::ostream &FROM_STATE_ACTION_SWITCH( int level ); + std::ostream &FINISH_CASES(); + std::ostream &EOF_ACTION_SWITCH( int level ); + unsigned int TO_STATE_ACTION( RedStateAp *state ); + unsigned int FROM_STATE_ACTION( RedStateAp *state ); + unsigned int EOF_ACTION( RedStateAp *state ); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/contrib/tools/ragel6/goflat.cpp b/contrib/tools/ragel6/goflat.cpp new file mode 100644 index 0000000000..bc7f51c6e7 --- /dev/null +++ b/contrib/tools/ragel6/goflat.cpp @@ -0,0 +1,764 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "goflat.h" +#include "redfsm.h" +#include "gendata.h" + +using std::endl; + +std::ostream &GoFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &GoFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &GoFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + +std::ostream &GoFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &GoFlatCodeGen::TO_STATE_ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GoFlatCodeGen::FROM_STATE_ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GoFlatCodeGen::EOF_ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, true, false ); + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &GoFlatCodeGen::ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &GoFlatCodeGen::FLAT_INDEX_OFFSET() +{ + out << " "; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + + /* Move the index offset ahead. */ + if ( st->transList != 0 ) + curIndOffset += keyOps->span( st->lowKey, st->highKey ); + + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << endl; + return out; +} + +std::ostream &GoFlatCodeGen::KEY_SPANS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->transList != 0 ) + span = keyOps->span( st->lowKey, st->highKey ); + out << span << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + +std::ostream &GoFlatCodeGen::TO_STATE_ACTIONS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + +std::ostream &GoFlatCodeGen::FROM_STATE_ACTIONS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + +std::ostream &GoFlatCodeGen::EOF_ACTIONS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + +std::ostream &GoFlatCodeGen::EOF_TRANS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans << ", "; + + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + + +std::ostream &GoFlatCodeGen::COND_KEYS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just cond low key and cond high key. */ + out << KEY( st->condLowKey ) << ", "; + out << KEY( st->condHighKey ) << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + + out << endl; + return out; +} + +std::ostream &GoFlatCodeGen::COND_KEY_SPANS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->condList != 0 ) + span = keyOps->span( st->condLowKey, st->condHighKey ); + out << span << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + +std::ostream &GoFlatCodeGen::CONDS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->condList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + if ( st->condList[pos] != 0 ) + out << st->condList[pos]->condSpaceId + 1 << ", "; + else + out << "0, "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + } + } + + out << endl; + return out; +} + +std::ostream &GoFlatCodeGen::COND_INDEX_OFFSET() +{ + out << " "; + int totalStateNum = 0; + int curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + + /* Move the index offset ahead. */ + if ( st->condList != 0 ) + curIndOffset += keyOps->span( st->condLowKey, st->condHighKey ); + } + out << endl; + return out; +} + + +std::ostream &GoFlatCodeGen::KEYS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just low key and high key. */ + out << KEY( st->lowKey ) << ", "; + out << KEY( st->highKey ) << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + + out << endl; + return out; +} + +std::ostream &GoFlatCodeGen::INDICIES() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->transList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + out << st->transList[pos]->id << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + out << st->defTrans->id << ", "; + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + + out << endl; + return out; +} + +std::ostream &GoFlatCodeGen::TRANS_TARGS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << " "; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Save the position. Needed for eofTargs. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id << ", "; + if ( t < redFsm->transSet.length()-1 ) { + if ( ++totalStates % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + delete[] transPtrs; + return out; +} + + +std::ostream &GoFlatCodeGen::TRANS_ACTIONS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << " "; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + out << ", "; + if ( t < redFsm->transSet.length()-1 ) { + if ( ++totalAct % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + delete[] transPtrs; + return out; +} + +void GoFlatCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << CAST(INT(), vCS() + " << 1") << endl << + " _inds = " << CAST(INT(), IO() + "[" + vCS() + "]") << endl << + endl << + " _slen = " << CAST(INT(), SP() + "[" + vCS() + "]") << endl << + " if _slen > 0 && " << K() << "[_keys] <= " << GET_WIDE_KEY() << " && " << + GET_WIDE_KEY() << " <= " << K() << "[_keys + 1]" << " {" << endl << + " _trans = " << CAST(INT(), I() + "[_inds + " + CAST(INT(), GET_WIDE_KEY() + " - " + K() + "[_keys]") + "]") << endl << + " } else {" << endl << + " _trans = " << CAST(INT(), I() + "[_inds + _slen]") << endl << + " }" << endl << + endl; +} + +void GoFlatCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + endl; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + endl; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + endl; + } + + STATE_IDS(); +} + +void GoFlatCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << CAST(WIDE_ALPH_TYPE(), GET_KEY()) << endl; + + out << + " _keys = " << CAST(INT(), vCS() + " << 1") << endl << + " _conds = " << CAST(INT(), CO() + "[" + vCS() + "]") << endl << + endl << + " _slen = " << CAST(INT(), CSP() + "[" + vCS() + "]") << endl << + " if _slen > 0 && " << CK() << "[_keys]" << " <= " << GET_WIDE_KEY() << " && " << + GET_WIDE_KEY() << " <= " << CK() << "[_keys + 1] {" << endl << + " _cond = " << CAST(INT(), C() + "[_conds + " + CAST(INT(), GET_WIDE_KEY() + " - " + CK() + "[_keys]") + "]") << endl << + " } else {" << endl << + " _cond = 0" << endl << + " }" << endl << + endl; + + out << + " switch _cond {" << endl; + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId + 1 << ":" << endl; + out << TABS(2) << "_widec = " << + KEY(condSpace->baseKey) << " + (" << CAST(WIDE_ALPH_TYPE(), GET_KEY()) << + " - " << KEY(keyOps->minKey) << ")" << endl; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " {" << endl << + " _widec += " << condValOffset << endl << + " }" << endl; + } + } + + out << + " }" << endl; +} + +void GoFlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {" << endl << + " var _slen " << INT() << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " var _ps " << INT() << endl; + + out << + " var _trans " << INT() << endl; + + if ( redFsm->anyConditions() ) + out << " var _cond " << INT() << endl; + + if ( redFsm->anyToStateActions() || + redFsm->anyRegActions() || redFsm->anyFromStateActions() ) + { + out << + " var _acts " << INT() << endl << + " var _nacts " << UINT() << endl; + } + + out << + " var _keys " << INT() << endl << + " var _inds " << INT() << endl; + + if ( redFsm->anyConditions() ) { + out << + " var _conds " << INT() << endl << + " var _widec " << WIDE_ALPH_TYPE() << endl; + } + + out << endl; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " == " << PE() << " {" << endl << + " goto _test_eof" << endl << + " }" << endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + out << "_resume:" << endl; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << CAST(INT(), FSA() + "[" + vCS() + "]") << endl << + " _nacts = " << CAST(UINT(), A() + "[_acts]") << "; _acts++" << endl << + " for ; _nacts > 0; _nacts-- {" << endl << + " _acts++" << endl << + " switch " << A() << "[_acts - 1]" << " {" << endl; + FROM_STATE_ACTION_SWITCH(2); + out << + " }" << endl << + " }" << endl << + endl; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:" << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << endl; + + out << + " " << vCS() << " = " << CAST(INT(), TT() + "[_trans]") << endl << + endl; + + if ( redFsm->anyRegActions() ) { + out << + " if " << TA() << "[_trans] == 0 {" << endl << + " goto _again" << endl << + " }" << endl << + endl << + " _acts = " << CAST(INT(), TA() + "[_trans]") << endl << + " _nacts = " << CAST(UINT(), A() + "[_acts]") << "; _acts++" << endl << + " for ; _nacts > 0; _nacts-- {" << endl << + " _acts++" << endl << + " switch " << A() << "[_acts - 1]" << " {" << endl; + ACTION_SWITCH(2); + out << + " }" << endl << + " }" << endl << + endl; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:" << endl; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << CAST(INT(), TSA() + "[" + vCS() + "]") << endl << + " _nacts = " << CAST(UINT(), A() + "[_acts]") << "; _acts++" << endl << + " for ; _nacts > 0; _nacts-- {" << endl << + " _acts++" << endl << + " switch " << A() << "[_acts - 1]" << " {" << endl; + TO_STATE_ACTION_SWITCH(2); + out << + " }" << endl << + " }" << endl << + endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + if ( !noEnd ) { + out << + " if " << P() << "++; " << P() << " != " << PE() << " {" << endl << + " goto _resume" << endl << + " }" << endl; + } + else { + out << + " " << P() << "++" << endl << + " goto _resume" << endl; + } + + if ( testEofUsed ) + out << " _test_eof: {}" << endl; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << " {" << endl; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0 {" << endl << + " _trans = " << CAST(INT(), ET() + "[" + vCS() + "] - 1") << endl << + " goto _eof_trans" << endl << + " }" << endl; + } + + if ( redFsm->anyEofActions() ) { + out << + " __acts := " << CAST(INT(), EA() + "[" + vCS() + "]") << endl << + " __nacts := " << CAST(UINT(), A() + "[__acts]") << "; __acts++" << endl << + " for ; __nacts > 0; __nacts-- {" << endl << + " __acts++" << endl << + " switch " << A() << "[__acts - 1]" << " {" << endl; + EOF_ACTION_SWITCH(3); + out << + " }" << endl << + " }" << endl; + } + + out << + " }" << endl << + endl; + } + + if ( outLabelUsed ) + out << " _out: {}" << endl; + + out << " }" << endl; +} diff --git a/contrib/tools/ragel6/goflat.h b/contrib/tools/ragel6/goflat.h new file mode 100644 index 0000000000..5216e85c75 --- /dev/null +++ b/contrib/tools/ragel6/goflat.h @@ -0,0 +1,80 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOFLAT_H +#define _GOFLAT_H + +#include <iostream> +#include "gotablish.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; + +/* + * GoFlatCodeGen + */ +class GoFlatCodeGen + : public GoTablishCodeGen +{ +public: + GoFlatCodeGen( ostream &out ) + : GoTablishCodeGen(out) {} + + virtual ~GoFlatCodeGen() { } + +protected: + std::ostream &TO_STATE_ACTION_SWITCH( int level ); + std::ostream &FROM_STATE_ACTION_SWITCH( int level ); + std::ostream &EOF_ACTION_SWITCH( int level ); + std::ostream &ACTION_SWITCH( int level ); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &FLAT_INDEX_OFFSET(); + std::ostream &KEY_SPANS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + void LOCATE_TRANS(); + + std::ostream &COND_INDEX_OFFSET(); + void COND_TRANSLATE(); + std::ostream &CONDS(); + std::ostream &COND_KEYS(); + std::ostream &COND_KEY_SPANS(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/contrib/tools/ragel6/goftable.cpp b/contrib/tools/ragel6/goftable.cpp new file mode 100644 index 0000000000..318f8783c9 --- /dev/null +++ b/contrib/tools/ragel6/goftable.cpp @@ -0,0 +1,441 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "goftable.h" +#include "redfsm.h" +#include "gendata.h" + +using std::endl; + +/* Determine if we should use indicies or not. */ +void GoFTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActListId) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActListId) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &GoFTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &GoFTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &GoFTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + + +/* Write out the function for a transition. */ +std::ostream &GoFTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &GoFTabCodeGen::TO_STATE_ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << endl; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &GoFTabCodeGen::FROM_STATE_ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << endl; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GoFTabCodeGen::EOF_ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true, false ); + + out << endl; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &GoFTabCodeGen::ACTION_SWITCH( int level ) +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << TABS(level) << "case " << redAct->actListId+1 << ":" << endl; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false, false ); + + out << endl; + } + } + + genLineDirective( out ); + return out; +} + +void GoFTabCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + endl; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + endl; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + endl; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + endl; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + endl; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + endl; + } + + STATE_IDS(); +} + +void GoFTabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {" << endl << + " var _klen " << INT() << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " var _ps " << INT() << endl; + + out << + " var _keys " << INT() << endl << + " var _trans " << INT() << endl; + + if ( redFsm->anyConditions() ) + out << " var _widec " << WIDE_ALPH_TYPE() << endl; + + out << endl; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " == " << PE() << " {" << endl << + " goto _test_eof" << endl << + " }" << endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + out << "_resume:" << endl; + + if ( redFsm->anyFromStateActions() ) { + out << + " switch " << FSA() << "[" << vCS() << "] {" << endl; + FROM_STATE_ACTION_SWITCH(1); + out << + " }" << endl << + endl; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + out << "_match:" << endl; + + if ( useIndicies ) + out << " _trans = " << CAST(INT(), I() + "[_trans]") << endl; + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:" << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << endl; + + out << + " " << vCS() << " = " << CAST(INT(), TT() + "[_trans]") << endl << + endl; + + if ( redFsm->anyRegActions() ) { + out << + " if " << TA() << "[_trans] == 0 {" << endl << + " goto _again" << endl << + " }" << endl << + endl << + " switch " << TA() << "[_trans] {" << endl; + ACTION_SWITCH(1); + out << + " }" << endl << + endl; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:" << endl; + + if ( redFsm->anyToStateActions() ) { + out << + " switch " << TSA() << "[" << vCS() << "] {" << endl; + TO_STATE_ACTION_SWITCH(1); + out << + " }" << endl << + endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + if ( !noEnd ) { + out << + " if " << P() << "++; " << P() << " != " << PE() << " {" << endl << + " goto _resume" << endl << + " }" << endl; + } + else { + out << + " " << P() << "++" << endl << + " goto _resume" << endl; + } + + if ( testEofUsed ) + out << " _test_eof: {}" << endl; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << " {" << endl; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0 {" << endl << + " _trans = " << CAST(INT(), ET() + "[" + vCS() + "] - 1") << endl << + " goto _eof_trans" << endl << + " }" << endl; + } + + if ( redFsm->anyEofActions() ) { + out << + " switch " << EA() << "[" << vCS() << "] {" << endl; + EOF_ACTION_SWITCH(2); + out << + " }" << endl; + } + + out << + " }" << endl << + endl; + } + + if ( outLabelUsed ) + out << " _out: {}" << endl; + + out << " }" << endl; +} diff --git a/contrib/tools/ragel6/goftable.h b/contrib/tools/ragel6/goftable.h new file mode 100644 index 0000000000..524b6a2174 --- /dev/null +++ b/contrib/tools/ragel6/goftable.h @@ -0,0 +1,59 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOFTABLE_H +#define _GOFTABLE_H + +#include <iostream> +#include "gotable.h" + +/* Forwards. */ +struct CodeGenData; + +/* + * GoFTabCode + */ +class GoFTabCodeGen + : public GoTabCodeGen +{ +public: + GoFTabCodeGen( ostream &out ) + : GoTabCodeGen(out) {} + +protected: + std::ostream &TO_STATE_ACTION_SWITCH( int level ); + std::ostream &FROM_STATE_ACTION_SWITCH( int level ); + std::ostream &EOF_ACTION_SWITCH( int level ); + std::ostream &ACTION_SWITCH( int level ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); + virtual void calcIndexSize(); +}; + +#endif diff --git a/contrib/tools/ragel6/gogoto.cpp b/contrib/tools/ragel6/gogoto.cpp new file mode 100644 index 0000000000..757308afa1 --- /dev/null +++ b/contrib/tools/ragel6/gogoto.cpp @@ -0,0 +1,734 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "gogoto.h" +#include "redfsm.h" +#include "bstmap.h" +#include "gendata.h" + +using std::endl; + +/* Emit the goto to take for a given transition. */ +std::ostream &GoGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + out << TABS(level) << "goto tr" << trans->id << ";"; + return out; +} + +int GoGotoCodeGen::TRANS_NR( RedTransAp *trans ) +{ + return trans->id; +} + +std::ostream &GoGotoCodeGen::TO_STATE_ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GoGotoCodeGen::FROM_STATE_ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GoGotoCodeGen::EOF_ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, true, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GoGotoCodeGen::ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +void GoGotoCodeGen::GOTO_HEADER( RedStateAp *state, int level ) +{ + /* Label the state. */ + out << TABS(level) << "case " << state->id << ":" << endl; +} + +void GoGotoCodeGen::emitSingleSwitch( RedStateAp *state, int level ) +{ + /* Load up the singles. */ + int numSingles = state->outSingle.length(); + RedTransEl *data = state->outSingle.data; + + if ( numSingles == 1 ) { + /* If there is a single single key then write it out as an if. */ + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " == " << + WIDE_KEY(state, data[0].lowKey) << " {" << endl; + + /* Virtual function for writing the target of the transition. */ + TRANS_GOTO(data[0].value, level + 1) << endl; + out << TABS(level) << "}" << endl; + } + else if ( numSingles > 1 ) { + /* Write out single keys in a switch if there is more than one. */ + out << TABS(level) << "switch " << GET_WIDE_KEY(state) << " {" << endl; + + /* Write out the single indicies. */ + for ( int j = 0; j < numSingles; j++ ) { + out << TABS(level) << "case " << WIDE_KEY(state, data[j].lowKey) << ":" << endl; + TRANS_GOTO(data[j].value, level + 1) << endl; + } + + /* Close off the transition switch. */ + out << TABS(level) << "}" << endl; + } +} + +void GoGotoCodeGen::emitRangeBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + RedTransEl *data = state->outRange.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid].lowKey == keyOps->minKey; + bool limitHigh = data[mid].highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "switch {" << endl; + out << TABS(level) << "case " << GET_WIDE_KEY(state) << " < " << + WIDE_KEY(state, data[mid].lowKey) << ":" << endl; + emitRangeBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "case " << GET_WIDE_KEY(state) << " > " << + WIDE_KEY(state, data[mid].highKey) << ":" << endl; + emitRangeBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "default:" << endl; + TRANS_GOTO(data[mid].value, level+1) << endl; + out << TABS(level) << "}" << endl; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "switch {" << endl; + out << TABS(level) << "case " << GET_WIDE_KEY(state) << " < " << + WIDE_KEY(state, data[mid].lowKey) << ":" << endl; + emitRangeBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "default:" << endl; + TRANS_GOTO(data[mid].value, level+1) << endl; + } + else { + out << TABS(level) << "case " << GET_WIDE_KEY(state) << " <= " << + WIDE_KEY(state, data[mid].highKey) << ":" << endl; + TRANS_GOTO(data[mid].value, level+1) << endl; + } + out << TABS(level) << "}" << endl; + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "switch {" << endl; + out << TABS(level) << "case " << GET_WIDE_KEY(state) << " > " << + WIDE_KEY(state, data[mid].highKey) << ":" << endl; + emitRangeBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "default:" << endl; + TRANS_GOTO(data[mid].value, level+1) << endl; + } + else { + out << TABS(level) << "case " << GET_WIDE_KEY(state) << " >= " << + WIDE_KEY(state, data[mid].lowKey) << ":" << endl; + TRANS_GOTO(data[mid].value, level+1) << endl; + } + out << TABS(level) << "}" << endl; + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if " << WIDE_KEY(state, data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " && " << GET_WIDE_KEY(state) << " <= " << + WIDE_KEY(state, data[mid].highKey) << " {" << endl; + TRANS_GOTO(data[mid].value, level+1) << endl; + out << TABS(level) << "}" << endl; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " <= " << + WIDE_KEY(state, data[mid].highKey) << " {" << endl; + TRANS_GOTO(data[mid].value, level+1) << endl; + out << TABS(level) << "}" << endl; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if " << WIDE_KEY(state, data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " {" << endl; + TRANS_GOTO(data[mid].value, level+1) << endl; + out << TABS(level) << "}" << endl; + } + else { + /* Both high and low are at the limit. No tests to do. */ + TRANS_GOTO(data[mid].value, level) << endl; + } + } +} + +void GoGotoCodeGen::STATE_GOTO_ERROR( int level ) +{ + /* Label the state and bail immediately. */ + outLabelUsed = true; + RedStateAp *state = redFsm->errState; + out << TABS(level) << "case " << state->id << ":" << endl; + out << TABS(level + 1) << "goto _out" << endl; +} + +void GoGotoCodeGen::COND_TRANSLATE( GenStateCond *stateCond, int level ) +{ + GenCondSpace *condSpace = stateCond->condSpace; + out << TABS(level) << "_widec = " << + KEY(condSpace->baseKey) << " + (" << CAST(WIDE_ALPH_TYPE(), GET_KEY()) << + " - " << KEY(keyOps->minKey) << ")" << endl; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(level) << "if "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " {" << endl; + out << TABS(level + 1) << "_widec += " << condValOffset << endl; + out << TABS(level) << "}" << endl; + } +} + +void GoGotoCodeGen::emitCondBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + GenStateCond **data = state->stateCondVect.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid]->lowKey == keyOps->minKey; + bool limitHigh = data[mid]->highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "switch {" << endl; + out << TABS(level) << "case " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << ":" << endl; + emitCondBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "case " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << ":" << endl; + emitCondBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "default:" << endl; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}" << endl; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "switch {" << endl; + out << TABS(level) << "case " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << ":" << endl; + emitCondBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "default:" << endl; + COND_TRANSLATE(data[mid], level+1); + } + else { + out << TABS(level) << "case " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << ":" << endl; + COND_TRANSLATE(data[mid], level+1); + } + out << TABS(level) << "}" << endl; + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "switch {" << endl; + out << TABS(level) << "case " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << ":" << endl; + emitCondBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "default:" << endl; + COND_TRANSLATE(data[mid], level+1); + } + else { + out << TABS(level) << "case " << GET_KEY() << " >= " << + KEY(data[mid]->lowKey) << ":" << endl; + COND_TRANSLATE(data[mid], level+1); + } + out << TABS(level) << "}" << endl; + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " && " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " {" << endl; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}" << endl; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " {" << endl; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}" << endl; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " {" << endl; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}" << endl; + } + else { + /* Both high and low are at the limit. No tests to do. */ + COND_TRANSLATE(data[mid], level); + } + } +} + +std::ostream &GoGotoCodeGen::STATE_GOTOS( int level ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(level); + else { + /* Writing code above state gotos. */ + GOTO_HEADER( st, level ); + + if ( st->stateCondVect.length() > 0 ) { + out << TABS(level + 1) << "_widec = " << CAST(WIDE_ALPH_TYPE(), GET_KEY()) << endl; + emitCondBSearch( st, level + 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st, level + 1 ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, level + 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, level + 1 ) << endl; + } + } + return out; +} + +std::ostream &GoGotoCodeGen::TRANSITIONS() +{ + /* Emit any transitions that have functions and that go to + * this state. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Write the label for the transition so it can be jumped to. */ + out << " tr" << trans->id << ": "; + + /* Destination state. */ + if ( trans->action != 0 && trans->action->anyCurStateRef() ) + out << "_ps = " << vCS() << ";"; + out << vCS() << " = " << trans->targ->id << "; "; + + if ( trans->action != 0 ) { + /* Write out the transition func. */ + out << "goto f" << trans->action->actListId << endl; + } + else { + /* No code to execute, just loop around. */ + out << "goto _again" << endl; + } + } + return out; +} + +std::ostream &GoGotoCodeGen::EXEC_FUNCS() +{ + /* Make labels that set acts and jump to execFuncs. Loop func indicies. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + out << " f" << redAct->actListId << ": " << + "_acts = " << (redAct->location + 1) << ";" + " goto execFuncs" << endl; + } + } + + out << + endl << + "execFuncs:" << endl << + " _nacts = " << CAST(UINT(), A() + "[_acts]") << "; _acts++" << endl << + " for ; _nacts > 0; _nacts-- {" << endl << + " _acts++" << endl << + " switch " << A() << "[_acts - 1]" << " {" << endl; + ACTION_SWITCH(2); + out << + " }" << endl << + " }" << endl << + " goto _again" << endl; + return out; +} + +unsigned int GoGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +unsigned int GoGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +unsigned int GoGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + +std::ostream &GoGotoCodeGen::TO_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = TO_STATE_ACTION(st); + + out << " "; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st] << ", "; + if ( st < numStates-1 ) { + if ( (st+1) % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + delete[] vals; + return out; +} + +std::ostream &GoGotoCodeGen::FROM_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = FROM_STATE_ACTION(st); + + out << " "; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st] << ", "; + if ( st < numStates-1 ) { + if ( (st+1) % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + delete[] vals; + return out; +} + +std::ostream &GoGotoCodeGen::EOF_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = EOF_ACTION(st); + + out << " "; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st] << ", "; + if ( st < numStates-1 ) { + if ( (st+1) % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + delete[] vals; + return out; +} + +std::ostream &GoGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << TABS(2) << "case " << st->id << ":" << endl; + + /* Write the goto func. */ + out << TABS(3) << "goto f" << st->eofAction->actListId << endl; + } + } + + return out; +} + +void GoGotoCodeGen::writeData() +{ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + endl; + } + + STATE_IDS(); +} + +void GoGotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " {" << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " var _ps " << INT() << " = 0" << endl; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " var _acts " << INT() << endl << + " var _nacts " << UINT() << endl; + } + + if ( redFsm->anyConditions() ) + out << " var _widec " << WIDE_ALPH_TYPE() << endl; + + out << endl; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " == " << PE() << " {" << endl << + " goto _test_eof" << endl << + " }" << endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + out << "_resume:" << endl; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << CAST(INT(), FSA() + "[" + vCS() + "]") << endl << + " _nacts = " << CAST(UINT(), A() + "[_acts]") << "; _acts++" << endl << + " for ; _nacts > 0; _nacts-- {" << endl << + " _acts++" << endl << + " switch " << A() << "[_acts - 1]" << " {" << endl; + FROM_STATE_ACTION_SWITCH(2); + out << + " }" << endl << + " }" << endl << + endl; + } + + out << + " switch " << vCS() << " {" << endl; + STATE_GOTOS(1); + out << + " }" << endl << + endl; + TRANSITIONS() << + endl; + + if ( redFsm->anyRegActions() ) + EXEC_FUNCS() << endl; + + out << "_again:" << endl; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << CAST(INT(), TSA() + "[" + vCS() + "]") << endl << + " _nacts = " << CAST(UINT(), A() + "[_acts]") << "; _acts++" << endl << + " for ; _nacts > 0; _nacts-- {" << endl << + " _acts++" << endl << + " switch " << A() << "[_acts - 1]" << " {" << endl; + TO_STATE_ACTION_SWITCH(2); + out << + " }" << endl << + " }" << endl << + endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + if ( !noEnd ) { + out << + " if " << P() << "++; " << P() << " != " << PE() << " {" << endl << + " goto _resume" << endl << + " }" << endl; + } + else { + out << + " " << P() << "++" << endl << + " goto _resume" << endl; + } + + if ( testEofUsed ) + out << " _test_eof: {}" << endl; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << " {" << endl; + + if ( redFsm->anyEofTrans() ) { + out << + " switch " << vCS() << " {" << endl; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << + " case " << st->id << ":" << endl << + " goto tr" << st->eofTrans->id << endl; + } + + out << + " }" << endl; + } + + if ( redFsm->anyEofActions() ) { + out << + " __acts := " << CAST(INT(), EA() + "[" + vCS() + "]") << endl << + " __nacts := " << CAST(UINT(), A() + "[__acts]") << "; __acts++" << endl << + " for ; __nacts > 0; __nacts-- {" << endl << + " __acts++" << endl << + " switch " << A() << "[__acts - 1]" << " {" << endl; + EOF_ACTION_SWITCH(3); + out << + " }" << endl << + " }" << endl; + } + + out << + " }" << endl << + endl; + } + + if ( outLabelUsed ) + out << " _out: {}" << endl; + + out << " }" << endl; +} diff --git a/contrib/tools/ragel6/gogoto.h b/contrib/tools/ragel6/gogoto.h new file mode 100644 index 0000000000..18d058e54d --- /dev/null +++ b/contrib/tools/ragel6/gogoto.h @@ -0,0 +1,83 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOGOTO_H +#define _GOGOTO_H + +#include <iostream> +#include "gotablish.h" + +/* Forwards. */ +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; +struct GenStateCond; + +/* + * Goto driven fsm. + */ +class GoGotoCodeGen + : public GoTablishCodeGen +{ +public: + GoGotoCodeGen( ostream &out ) + : GoTablishCodeGen(out) {} + +protected: + std::ostream &TO_STATE_ACTION_SWITCH( int level ); + std::ostream &FROM_STATE_ACTION_SWITCH( int level ); + std::ostream &EOF_ACTION_SWITCH( int level ); + std::ostream &ACTION_SWITCH( int level ); + std::ostream &STATE_GOTOS( int level ); + std::ostream &TRANSITIONS(); + std::ostream &EXEC_FUNCS(); + std::ostream &FINISH_CASES(); + + virtual unsigned int TO_STATE_ACTION( RedStateAp *state ); + virtual unsigned int FROM_STATE_ACTION( RedStateAp *state ); + virtual unsigned int EOF_ACTION( RedStateAp *state ); + + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + + void COND_TRANSLATE( GenStateCond *stateCond, int level ); + void emitCondBSearch( RedStateAp *state, int level, int low, int high ); + void STATE_CONDS( RedStateAp *state, bool genDefault ); + + virtual std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + virtual int TRANS_NR( RedTransAp *trans ); + + void emitSingleSwitch( RedStateAp *state, int level ); + void emitRangeBSearch( RedStateAp *state, int level, int low, int high ); + + /* Called from STATE_GOTOS just before writing the gotos */ + virtual void GOTO_HEADER( RedStateAp *state, int level ); + virtual void STATE_GOTO_ERROR( int level ); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/contrib/tools/ragel6/goipgoto.cpp b/contrib/tools/ragel6/goipgoto.cpp new file mode 100644 index 0000000000..153197e049 --- /dev/null +++ b/contrib/tools/ragel6/goipgoto.cpp @@ -0,0 +1,477 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "goipgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +using std::endl; + +bool GoIpGotoCodeGen::useAgainLabel() +{ + return redFsm->anyRegActionRets() || + redFsm->anyRegActionByValControl() || + redFsm->anyRegNextStmt(); +} + +void GoIpGotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << "goto st" << gotoDest << " }"; +} + +void GoIpGotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "] = " << targState << + "; " << TOP() << "++; " << "goto st" << callDest << " }"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void GoIpGotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "] = " << targState << "; " << TOP() << "++; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << "); " << "goto _again }"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void GoIpGotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << TOP() << "--; " << vCS() << " = " << STACK() << "[" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false, false ); + ret << "}"; + } + + ret << "goto _again }"; +} + +void GoIpGotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << "); " << "goto _again }"; +} + +void GoIpGotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void GoIpGotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ");"; +} + +void GoIpGotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void GoIpGotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << targState; +} + +void GoIpGotoCodeGen::BREAK( ostream &ret, int targState, bool csForced ) +{ + outLabelUsed = true; + ret << "{" << P() << "++; "; + if ( !csForced ) + ret << vCS() << " = " << targState << "; "; + ret << "goto _out }"; +} + +bool GoIpGotoCodeGen::IN_TRANS_ACTIONS( RedStateAp *state ) +{ + bool anyWritten = false; + + /* Emit any transitions that have actions and that go to this state. */ + for ( int it = 0; it < state->numInTrans; it++ ) { + RedTransAp *trans = state->inTrans[it]; + if ( trans->action != 0 && trans->labelNeeded ) { + /* Remember that we wrote an action so we know to write the + * line directive for going back to the output. */ + anyWritten = true; + + /* Write the label for the transition so it can be jumped to. */ + out << "tr" << trans->id << ":" << endl; + + /* If the action contains a next, then we must preload the current + * state since the action may or may not set it. */ + if ( trans->action->anyNextStmt() ) + out << " " << vCS() << " = " << trans->targ->id << endl; + + /* Write each action in the list. */ + for ( GenActionTable::Iter item = trans->action->key; item.lte(); item++ ) { + ACTION( out, item->value, trans->targ->id, false, + trans->action->anyNextStmt() ); + } + + /* If the action contains a next then we need to reload, otherwise + * jump directly to the target state. */ + if ( trans->action->anyNextStmt() ) + out << " goto _again" << endl; + else + out << " goto st" << trans->targ->id << endl; + } + } + + return anyWritten; +} + +std::ostream &GoIpGotoCodeGen::STATE_GOTOS_SWITCH( int level ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + out << TABS(level) << "case " << st->id << ":" << endl; + out << TABS(level + 1) << "goto st_case_" << st->id << endl; + } + return out; +} + +/* Called from GotoCodeGen::STATE_GOTOS just before writing the gotos for each + * state. */ +void GoIpGotoCodeGen::GOTO_HEADER( RedStateAp *state, int level ) +{ + bool anyWritten = IN_TRANS_ACTIONS( state ); + + if ( state->labelNeeded ) + out << TABS(level) << "st" << state->id << ":" << endl; + + if ( state->toStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->toStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, state->id, false, + state->toStateAction->anyNextStmt() ); + } + } + + /* Advance and test buffer pos. */ + if ( state->labelNeeded ) { + if ( !noEnd ) { + out << + TABS(level + 1) << "if " << P() << "++; " << P() << " == " << PE() << " {" << endl << + TABS(level + 2) << "goto _test_eof" << state->id << endl << + TABS(level + 1) << "}" << endl; + } + else { + out << + TABS(level + 1) << P() << "++" << endl; + } + } + + /* Give the state a label. */ + out << TABS(level) << "st_case_" << state->id << ":" << endl; + + if ( state->fromStateAction != 0 ) { + /* Remember that we wrote an action. Write every action in the list. */ + anyWritten = true; + for ( GenActionTable::Iter item = state->fromStateAction->key; item.lte(); item++ ) { + ACTION( out, item->value, state->id, false, + state->fromStateAction->anyNextStmt() ); + } + } + + if ( anyWritten ) + genLineDirective( out ); + + /* Record the prev state if necessary. */ + if ( state->anyRegCurStateRef() ) + out << TABS(level + 1) << "_ps = " << state->id << endl; +} + +void GoIpGotoCodeGen::STATE_GOTO_ERROR( int level ) +{ + /* In the error state we need to emit some stuff that usually goes into + * the header. */ + RedStateAp *state = redFsm->errState; + bool anyWritten = IN_TRANS_ACTIONS( state ); + + /* No case label needed since we don't switch on the error state. */ + if ( anyWritten ) + genLineDirective( out ); + + out << "st_case_" << state->id << ":" << endl; + if ( state->labelNeeded ) + out << TABS(level) << "st" << state->id << ":" << endl; + + /* Break out here. */ + outLabelUsed = true; + out << TABS(level + 1) << vCS() << " = " << state->id << endl; + out << TABS(level + 1) << "goto _out" << endl; +} + + +/* Emit the goto to take for a given transition. */ +std::ostream &GoIpGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + out << TABS(level) << "goto tr" << trans->id; + } + else { + /* Go directly to the target state. */ + out << TABS(level) << "goto st" << trans->targ->id; + } + return out; +} + +int GoIpGotoCodeGen::TRANS_NR( RedTransAp *trans ) +{ + if ( trans->action != 0 ) { + /* Go to the transition which will go to the state. */ + return trans->id + redFsm->stateList.length(); + } + else { + /* Go directly to the target state. */ + return trans->targ->id; + } +} + +std::ostream &GoIpGotoCodeGen::EXIT_STATES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->outNeeded ) { + testEofUsed = true; + out << " _test_eof" << st->id << ": " << vCS() << " = " << + st->id << "; goto _test_eof" << endl; + } + } + return out; +} + +std::ostream &GoIpGotoCodeGen::AGAIN_CASES( int level ) +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + out << + TABS(level) << "case " << st->id << ":" << endl << + TABS(level + 1) << "goto st" << st->id << endl; + } + return out; +} + +std::ostream &GoIpGotoCodeGen::FINISH_CASES( int level ) +{ + bool anyWritten = false; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofAction != 0 ) { + if ( st->eofAction->eofRefs == 0 ) + st->eofAction->eofRefs = new IntSet; + st->eofAction->eofRefs->insert( st->id ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << TABS(level) << "case " << st->id << ":" << endl << + TABS(level + 1) << "goto tr" << st->eofTrans->id << endl; + } + + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + if ( act->eofRefs != 0 ) { + out << TABS(level) << "case "; + for ( IntSet::Iter pst = *act->eofRefs; pst.lte(); pst++ ) { + out << *pst; + if ( !pst.last() ) + out << ", "; + } + out << ":" << endl; + + /* Remember that we wrote a trans so we know to write the + * line directive for going back to the output. */ + anyWritten = true; + + /* Write each action in the eof action list. */ + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) + ACTION( out, item->value, STATE_ERR_STATE, true, false ); + } + } + + if ( anyWritten ) + genLineDirective( out ); + return out; +} + +void GoIpGotoCodeGen::setLabelsNeeded( GenInlineList *inlineList ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Goto: case GenInlineItem::Call: { + /* Mark the target as needing a label. */ + item->targState->labelNeeded = true; + break; + } + default: break; + } + + if ( item->children != 0 ) + setLabelsNeeded( item->children ); + } +} + +/* Set up labelNeeded flag for each state. */ +void GoIpGotoCodeGen::setLabelsNeeded() +{ + /* If we use the _again label, then we the _again switch, which uses all + * labels. */ + if ( useAgainLabel() ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = true; + } + else { + /* Do not use all labels by default, init all labelNeeded vars to false. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + st->labelNeeded = false; + + /* Walk all transitions and set only those that have targs. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* If there is no action with a next statement, then the label will be + * needed. */ + if ( trans->action == 0 || !trans->action->anyNextStmt() ) + trans->targ->labelNeeded = true; + + /* Need labels for states that have goto or calls in action code + * invoked on characters (ie, not from out action code). */ + if ( trans->action != 0 ) { + /* Loop the actions. */ + for ( GenActionTable::Iter act = trans->action->key; act.lte(); act++ ) { + /* Get the action and walk it's tree. */ + setLabelsNeeded( act->value->inlineList ); + } + } + } + } + + if ( !noEnd ) { + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st != redFsm->errState ) + st->outNeeded = st->labelNeeded; + } + } +} + +void GoIpGotoCodeGen::writeData() +{ + STATE_IDS(); +} + +void GoIpGotoCodeGen::writeExec() +{ + /* Must set labels immediately before writing because we may depend on the + * noend write option. */ + setLabelsNeeded(); + testEofUsed = false; + outLabelUsed = false; + + out << " {" << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " var _ps " << INT() << " = 0" << endl; + + if ( redFsm->anyConditions() ) + out << " var _widec " << WIDE_ALPH_TYPE() << endl; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " == " << PE() << " {" << endl << + " goto _test_eof" << endl << + " }" << endl; + } + + if ( useAgainLabel() ) { + out << + " goto _resume" << endl << + endl << + "_again:" << endl << + " switch " << vCS() << " {" << endl; + AGAIN_CASES(1) << + " }" << endl << + endl; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << "++; " << P() << " == " << PE() << " {" << endl << + " goto _test_eof" << endl << + " }" << endl; + } + else { + out << + " " << P() << "++" << endl; + } + out << "_resume:" << endl; + } + + out << + " switch " << vCS() << " {" << endl; + STATE_GOTOS_SWITCH(1); + out << + " }" << endl; + out << " goto st_out" << endl; + STATE_GOTOS(1); + out << " st_out:" << endl; + EXIT_STATES() << + endl; + + if ( testEofUsed ) + out << " _test_eof: {}" << endl; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << " {" << endl << + " switch " << vCS() << " {" << endl; + FINISH_CASES(2); + out << + " }" << endl << + " }" << endl << + endl; + } + + if ( outLabelUsed ) + out << " _out: {}" << endl; + + out << + " }" << endl; +} diff --git a/contrib/tools/ragel6/goipgoto.h b/contrib/tools/ragel6/goipgoto.h new file mode 100644 index 0000000000..cceaee0c63 --- /dev/null +++ b/contrib/tools/ragel6/goipgoto.h @@ -0,0 +1,75 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOIPGOTO_H +#define _GOIPGOTO_H + +#include <iostream> +#include "gogoto.h" + +/* Forwards. */ +struct CodeGenData; + +class GoIpGotoCodeGen + : public GoGotoCodeGen +{ +public: + GoIpGotoCodeGen( ostream &out ) + : GoGotoCodeGen(out) {} + + std::ostream &EXIT_STATES(); + std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + int TRANS_NR( RedTransAp *trans ); + std::ostream &FINISH_CASES( int level ); + std::ostream &AGAIN_CASES( int level ); + std::ostream &STATE_GOTOS_SWITCH( int level ); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &ret, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void BREAK( ostream &ret, int targState, bool csForced ); + + virtual void writeData(); + virtual void writeExec(); + +protected: + bool useAgainLabel(); + + /* Called from GotoCodeGen::STATE_GOTOS just before writing the gotos for + * each state. */ + bool IN_TRANS_ACTIONS( RedStateAp *state ); + void GOTO_HEADER( RedStateAp *state, int level ); + void STATE_GOTO_ERROR( int level ); + + /* Set up labelNeeded flag for each state. */ + void setLabelsNeeded( GenInlineList *inlineList ); + void setLabelsNeeded(); +}; + +#endif diff --git a/contrib/tools/ragel6/gotable.cpp b/contrib/tools/ragel6/gotable.cpp new file mode 100644 index 0000000000..01152d39bf --- /dev/null +++ b/contrib/tools/ragel6/gotable.cpp @@ -0,0 +1,977 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <sstream> +#include "ragel.h" +#include "gotable.h" +#include "redfsm.h" +#include "gendata.h" + +using std::endl; + +/* Determine if we should use indicies or not. */ +void GoTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActionLoc) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActionLoc) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &GoTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &GoTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &GoTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + + +std::ostream &GoTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &GoTabCodeGen::TO_STATE_ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GoTabCodeGen::FROM_STATE_ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, false, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &GoTabCodeGen::EOF_ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, true, false ); + } + } + + genLineDirective(out); + return out; +} + + +std::ostream &GoTabCodeGen::ACTION_SWITCH( int level ) +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << TABS(level) << "case " << act->actionId << ":" << endl; + ACTION( out, act, 0, false, false ); + } + } + + genLineDirective(out); + return out; +} + +std::ostream &GoTabCodeGen::COND_OFFSETS() +{ + out << " "; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->stateCondList.length(); + } + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::KEY_OFFSETS() +{ + out << " "; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->outSingle.length() + st->outRange.length()*2; + } + out << endl; + return out; +} + + +std::ostream &GoTabCodeGen::INDEX_OFFSETS() +{ + out << " "; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + + /* Move the index offset ahead. */ + curIndOffset += st->outSingle.length() + st->outRange.length(); + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::COND_LENS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->stateCondList.length(); + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + + +std::ostream &GoTabCodeGen::SINGLE_LENS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->outSingle.length(); + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::RANGE_LENS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit length of range index. */ + out << st->outRange.length(); + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::TO_STATE_ACTIONS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::FROM_STATE_ACTIONS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::EOF_ACTIONS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::EOF_TRANS() +{ + out << " "; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + out << ", "; + if ( !st.last() ) { + if ( ++totalStateNum % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + return out; +} + + +std::ostream &GoTabCodeGen::COND_KEYS() +{ + out << " "; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Lower key. */ + out << KEY( sc->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + + /* Upper key. */ + out << KEY( sc->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + } + + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::COND_SPACES() +{ + out << " "; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Cond Space id. */ + out << sc->condSpace->condSpaceId << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + } + + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::KEYS() +{ + out << " "; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << KEY( stel->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + + /* Loop the state's transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + /* Lower key. */ + out << KEY( rtel->lowKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + + /* Upper key. */ + out << KEY( rtel->highKey ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + } + + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::INDICIES() +{ + out << " "; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << stel->value->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + out << rtel->value->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + out << st->defTrans->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + } + + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::TRANS_TARGS() +{ + out << " "; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + + /* The state's default target state. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + } + + /* Add any eof transitions that have not yet been written out above. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + trans->pos = totalTrans; + out << trans->targ->id << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + } + + + out << endl; + return out; +} + + +std::ostream &GoTabCodeGen::TRANS_ACTIONS() +{ + out << " "; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + } + + /* Add any eof transitions that have not yet been written out above. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + TRANS_ACTION( trans ) << ", "; + if ( ++totalTrans % IALL == 0 ) + out << endl << " "; + } + } + + out << endl; + return out; +} + +std::ostream &GoTabCodeGen::TRANS_TARGS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << " "; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Record the position, need this for eofTrans. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id << ", "; + if ( t < redFsm->transSet.length()-1 ) { + if ( ++totalStates % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + delete[] transPtrs; + return out; +} + + +std::ostream &GoTabCodeGen::TRANS_ACTIONS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << " "; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + out << ", "; + if ( t < redFsm->transSet.length()-1 ) { + if ( ++totalAct % IALL == 0 ) + out << endl << " "; + } + } + out << endl; + delete[] transPtrs; + return out; +} + +void GoTabCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << endl; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << endl; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << endl; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << endl; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << endl; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << endl; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << endl; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << endl; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << endl; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << endl; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << endl; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << endl; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << endl; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << endl; + } + + STATE_IDS(); +} + +void GoTabCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << CAST(INT(), KO() + "[" + vCS() + "]") << endl << + " _trans = " << CAST(INT(), IO() + "[" + vCS() + "]") << endl << + endl << + " _klen = " << CAST(INT(), SL() + "[" + vCS() + "]") << endl << + " if _klen > 0 {" << endl << + " _lower := " << CAST(INT(), "_keys") << endl << + " var _mid " << INT() << endl << + " _upper := " << CAST(INT(), "_keys + _klen - 1") << endl << + " for {" << endl << + " if _upper < _lower {" << endl << + " break" << endl << + " }" << endl << + endl << + " _mid = _lower + ((_upper - _lower) >> 1)" << endl << + " switch {" << endl << + " case " << GET_WIDE_KEY() << " < " << K() << "[_mid]" << ":" << endl << + " _upper = _mid - 1" << endl << + " case " << GET_WIDE_KEY() << " > " << K() << "[_mid]" << ":" << endl << + " _lower = _mid + 1" << endl << + " default:" << endl << + " _trans += " << CAST(INT(), "_mid - " + CAST(INT(), "_keys")) << endl << + " goto _match" << endl << + " }" << endl << + " }" << endl << + " _keys += _klen" << endl << + " _trans += _klen" << endl << + " }" << endl << + endl << + " _klen = " << CAST(INT(), RL() + "[" + vCS() + "]") << endl << + " if _klen > 0 {" << endl << + " _lower := " << CAST(INT(), "_keys") << endl << + " var _mid " << INT() << endl << + " _upper := " << CAST(INT(), "_keys + (_klen << 1) - 2") << endl << + " for {" << endl << + " if _upper < _lower {" << endl << + " break" << endl << + " }" << endl << + endl << + " _mid = _lower + (((_upper - _lower) >> 1) & ^1)" << endl << + " switch {" << endl << + " case " << GET_WIDE_KEY() << " < " << K() << "[_mid]" << ":" << endl << + " _upper = _mid - 2" << endl << + " case " << GET_WIDE_KEY() << " > " << K() << "[_mid + 1]" << ":" << endl << + " _lower = _mid + 2" << endl << + " default:" << endl << + " _trans += " << CAST(INT(), "(_mid - " + CAST(INT(), "_keys") + ") >> 1") << endl << + " goto _match" << endl << + " }" << endl << + " }" << endl << + " _trans += _klen" << endl << + " }" << endl << + endl; +} + +void GoTabCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << CAST(WIDE_ALPH_TYPE(), GET_KEY()) << endl << + " _klen = " << CAST(INT(), CL() + "[" + vCS() + "]") << endl << + " _keys = " << CAST(INT(), CO() + "[" + vCS() + "] * 2") << endl << + " if _klen > 0 {" << endl << + " _lower := " << CAST(INT(), "_keys") << endl << + " var _mid " << INT() << endl << + " _upper := " << CAST(INT(), "_keys + (_klen << 1) - 2") << endl << + " COND_LOOP:" << endl << + " for {" << endl << + " if _upper < _lower {" << endl << + " break" << endl << + " }" << endl << + endl << + " _mid = _lower + (((_upper - _lower) >> 1) & ^1)" << endl << + " switch {" << endl << + " case " << GET_WIDE_KEY() << " < " << CAST(WIDE_ALPH_TYPE(), CK() + "[_mid]") << ":" << endl << + " _upper = _mid - 2" << endl << + " case " << GET_WIDE_KEY() << " > " << CAST(WIDE_ALPH_TYPE(), CK() + "[_mid + 1]") << ":" << endl << + " _lower = _mid + 2" << endl << + " default:" << endl << + " switch " << C() << "[" << CAST(INT(), CO() + "[" + vCS() + "]") << + " + ((_mid - _keys)>>1)] {" << endl; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << TABS(4) << "case " << condSpace->condSpaceId << ":" << endl; + out << TABS(5) << "_widec = " << KEY(condSpace->baseKey) << " + (" << CAST(WIDE_ALPH_TYPE(), GET_KEY()) << + " - " << KEY(keyOps->minKey) << ")" << endl; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(5) << "if "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " {" << endl << TABS(6) << "_widec += " << condValOffset << endl << TABS(5) << "}" << endl; + } + } + + out << + " }" << endl << + " break COND_LOOP" << endl << + " }" << endl << + " }" << endl << + " }" << endl << + endl; +} + +void GoTabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << + " {" << endl << + " var _klen " << INT() << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " var _ps " << INT() << endl; + + out << + " var _trans " << INT() << endl; + + if ( redFsm->anyConditions() ) + out << " var _widec " << WIDE_ALPH_TYPE() << endl; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " var _acts " << INT() << endl << + " var _nacts " << UINT() << endl; + } + + out << + " var _keys " << INT() << endl; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " == " << PE() << " {" << endl << + " goto _test_eof" << endl << + " }" << endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + out << "_resume:" << endl; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << CAST(INT(), FSA() + "[" + vCS() + "]") << endl << + " _nacts = " << CAST(UINT(), A() + "[_acts]") << "; _acts++" << endl << + " for ; _nacts > 0; _nacts-- {" << endl << + " _acts++" << endl << + " switch " << A() << "[_acts - 1]" << " {" << endl; + FROM_STATE_ACTION_SWITCH(2); + out << + " }" << endl << + " }" << endl << endl; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + out << "_match:" << endl; + + if ( useIndicies ) + out << " _trans = " << CAST(INT(), I() + "[_trans]") << endl; + + if ( redFsm->anyEofTrans() ) + out << "_eof_trans:" << endl; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << endl; + + out << + " " << vCS() << " = " << CAST(INT(), TT() + "[_trans]") << endl << endl; + + if ( redFsm->anyRegActions() ) { + out << + " if " << TA() << "[_trans] == 0 {" << endl << + " goto _again" << endl << + " }" << endl << + endl << + " _acts = " << CAST(INT(), TA() + "[_trans]") << endl << + " _nacts = " << CAST(UINT(), A() + "[_acts]") << "; _acts++" << endl << + " for ; _nacts > 0; _nacts-- {" << endl << + " _acts++" << endl << + " switch " << A() << "[_acts-1]" << " {" << endl; + ACTION_SWITCH(2); + out << + " }" << endl << + " }" << endl << endl; + } + + if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || + redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "_again:" << endl; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << CAST(INT(), TSA() + "[" + vCS() + "]") << endl << + " _nacts = " << CAST(UINT(), A() + "[_acts]") << "; _acts++" << endl << + " for ; _nacts > 0; _nacts-- {" << endl << + " _acts++" << endl << + " switch " << A() << "[_acts-1] {" << endl; + TO_STATE_ACTION_SWITCH(2); + out << + " }" << endl << + " }" << endl << endl; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " == " << redFsm->errState->id << " {" << endl << + " goto _out" << endl << + " }" << endl; + } + + if ( !noEnd ) { + out << + " " << P() << "++" << endl << + " if " << P() << " != " << PE() << " {" << endl << + " goto _resume" << endl << + " }" << endl; + } + else { + out << + " " << P() << "++" << endl << + " goto _resume" << endl; + } + + if ( testEofUsed ) + out << " _test_eof: {}" << endl; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << " {" << endl; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0 {" << endl << + " _trans = " << CAST(INT(), ET() + "[" + vCS() + "] - 1") << endl << + " goto _eof_trans" << endl << + " }" << endl; + } + + if ( redFsm->anyEofActions() ) { + out << + " __acts := " << EA() << "[" << vCS() << "]" << endl << + " __nacts := " << CAST(UINT(), A() + "[__acts]") << "; __acts++" << endl << + " for ; __nacts > 0; __nacts-- {" << endl << + " __acts++" << endl << + " switch " << A() << "[__acts-1] {" << endl; + EOF_ACTION_SWITCH(3); + out << + " }" << endl << + " }" << endl; + } + + out << + " }" << endl << endl; + } + + if ( outLabelUsed ) + out << " _out: {}" << endl; + + out << " }" << endl; +} diff --git a/contrib/tools/ragel6/gotable.h b/contrib/tools/ragel6/gotable.h new file mode 100644 index 0000000000..a62b8a0906 --- /dev/null +++ b/contrib/tools/ragel6/gotable.h @@ -0,0 +1,76 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOTABLE_H +#define _GOTABLE_H + +#include <iostream> +#include "gotablish.h" + +class GoTabCodeGen + : public GoTablishCodeGen +{ +public: + GoTabCodeGen( ostream &out ) + : GoTablishCodeGen(out) {} + + virtual ~GoTabCodeGen() { } + virtual void writeData(); + virtual void writeExec(); + +protected: + std::ostream &TO_STATE_ACTION_SWITCH( int level ); + std::ostream &FROM_STATE_ACTION_SWITCH( int level ); + std::ostream &EOF_ACTION_SWITCH( int level ); + std::ostream &ACTION_SWITCH( int level ); + + std::ostream &COND_KEYS(); + std::ostream &COND_SPACES(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &COND_OFFSETS(); + std::ostream &KEY_OFFSETS(); + std::ostream &INDEX_OFFSETS(); + std::ostream &COND_LENS(); + std::ostream &SINGLE_LENS(); + std::ostream &RANGE_LENS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + std::ostream &TRANS_TARGS_WI(); + std::ostream &TRANS_ACTIONS_WI(); + void LOCATE_TRANS(); + + void COND_TRANSLATE(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void calcIndexSize(); +}; + +#endif diff --git a/contrib/tools/ragel6/gotablish.cpp b/contrib/tools/ragel6/gotablish.cpp new file mode 100644 index 0000000000..218ab47908 --- /dev/null +++ b/contrib/tools/ragel6/gotablish.cpp @@ -0,0 +1,111 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "gotablish.h" + +using std::endl; + +void GoTablishCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << vCS() << " = " << gotoDest << endl << + "goto _again" << endl; +} + +void GoTablishCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ")" << endl << "goto _again" << endl; +} + +void GoTablishCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void GoTablishCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void GoTablishCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << endl; +} + +void GoTablishCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish, false ); + ret << ")" << endl; +} + +void GoTablishCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{ "; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << STACK() << "[" << TOP() << "] = " << vCS() << "; " << TOP() << "++; " << + vCS() << " = " << callDest << "; " << "goto _again" << endl; + + if ( prePushExpr != 0 ) + ret << " }"; +} + +void GoTablishCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false, false ); + } + + ret << STACK() << "[" << TOP() << "] = " << vCS() << "; " << TOP() << "++; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish, false ); + ret << "); " << "goto _again" << endl; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void GoTablishCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << TOP() << "--; " << vCS() << " = " << STACK() << "[" << + TOP() << "]" << endl; + + if ( postPopExpr != 0 ) { + ret << "{ "; + INLINE_LIST( ret, postPopExpr, 0, false, false ); + ret << " }" << endl; + } + + ret << "goto _again" << endl; +} + +void GoTablishCodeGen::BREAK( ostream &ret, int targState, bool csForced ) +{ + outLabelUsed = true; + ret << P() << "++; " << "goto _out" << endl; +} diff --git a/contrib/tools/ragel6/gotablish.h b/contrib/tools/ragel6/gotablish.h new file mode 100644 index 0000000000..9e79b11796 --- /dev/null +++ b/contrib/tools/ragel6/gotablish.h @@ -0,0 +1,48 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _GOTABLISH_H +#define _GOTABLISH_H + +#include "gocodegen.h" + +class GoTablishCodeGen + : public GoCodeGen +{ +public: + GoTablishCodeGen( ostream &out ) + : GoCodeGen(out) {} +protected: + virtual void GOTO( ostream &ret, int gotoDest, bool inFinish ); + virtual void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + virtual void NEXT( ostream &ret, int nextDest, bool inFinish ); + virtual void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + virtual void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + virtual void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + virtual void CURS( ostream &ret, bool inFinish ); + virtual void TARGS( ostream &ret, bool inFinish, int targState ); + virtual void RET( ostream &ret, bool inFinish ); + virtual void BREAK( ostream &ret, int targState, bool csForced ); +}; + +#endif diff --git a/contrib/tools/ragel6/inputdata.cpp b/contrib/tools/ragel6/inputdata.cpp new file mode 100644 index 0000000000..7d8322ff70 --- /dev/null +++ b/contrib/tools/ragel6/inputdata.cpp @@ -0,0 +1,277 @@ +/* + * Copyright 2008 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "common.h" +#include "inputdata.h" +#include "parsedata.h" +#include "rlparse.h" +#include <iostream> +#include "dotcodegen.h" + +using std::cout; +using std::cerr; +using std::endl; +using std::ios; + +/* Invoked by the parser when the root element is opened. */ +void InputData::cdDefaultFileName( const char *inputFile ) +{ + /* If the output format is code and no output file name is given, then + * make a default. */ + if ( outputFileName == 0 ) { + const char *ext = findFileExtension( inputFile ); + if ( ext != 0 && strcmp( ext, ".rh" ) == 0 ) + outputFileName = fileNameFromStem( inputFile, ".h" ); + else { + const char *defExtension = 0; + switch ( hostLang->lang ) { + case HostLang::C: defExtension = ".c"; break; + case HostLang::D: defExtension = ".d"; break; + case HostLang::D2: defExtension = ".d"; break; + default: break; + } + outputFileName = fileNameFromStem( inputFile, defExtension ); + } + } +} + +/* Invoked by the parser when the root element is opened. */ +void InputData::goDefaultFileName( const char *inputFile ) +{ + /* If the output format is code and no output file name is given, then + * make a default. */ + if ( outputFileName == 0 ) + outputFileName = fileNameFromStem( inputFile, ".go" ); +} + +/* Invoked by the parser when the root element is opened. */ +void InputData::javaDefaultFileName( const char *inputFile ) +{ + /* If the output format is code and no output file name is given, then + * make a default. */ + if ( outputFileName == 0 ) + outputFileName = fileNameFromStem( inputFile, ".java" ); +} + +/* Invoked by the parser when the root element is opened. */ +void InputData::rubyDefaultFileName( const char *inputFile ) +{ + /* If the output format is code and no output file name is given, then + * make a default. */ + if ( outputFileName == 0 ) + outputFileName = fileNameFromStem( inputFile, ".rb" ); +} + +/* Invoked by the parser when the root element is opened. */ +void InputData::csharpDefaultFileName( const char *inputFile ) +{ + /* If the output format is code and no output file name is given, then + * make a default. */ + if ( outputFileName == 0 ) { + const char *ext = findFileExtension( inputFile ); + if ( ext != 0 && strcmp( ext, ".rh" ) == 0 ) + outputFileName = fileNameFromStem( inputFile, ".h" ); + else + outputFileName = fileNameFromStem( inputFile, ".cs" ); + } +} + +/* Invoked by the parser when the root element is opened. */ +void InputData::ocamlDefaultFileName( const char *inputFile ) +{ + /* If the output format is code and no output file name is given, then + * make a default. */ + if ( outputFileName == 0 ) + outputFileName = fileNameFromStem( inputFile, ".ml" ); +} + +void InputData::makeOutputStream() +{ + if ( ! generateDot && ! generateXML ) { + switch ( hostLang->lang ) { + case HostLang::C: + case HostLang::D: + case HostLang::D2: + cdDefaultFileName( inputFileName ); + break; + case HostLang::Java: + javaDefaultFileName( inputFileName ); + break; + case HostLang::Go: + goDefaultFileName( inputFileName ); + break; + case HostLang::Ruby: + rubyDefaultFileName( inputFileName ); + break; + case HostLang::CSharp: + csharpDefaultFileName( inputFileName ); + break; + case HostLang::OCaml: + ocamlDefaultFileName( inputFileName ); + break; + } + } + + /* Make sure we are not writing to the same file as the input file. */ + if ( outputFileName != 0 ) { + if ( strcmp( inputFileName, outputFileName ) == 0 ) { + error() << "output file \"" << outputFileName << + "\" is the same as the input file" << endl; + } + + /* Create the filter on the output and open it. */ + outFilter = new output_filter( outputFileName ); + + /* Open the output stream, attaching it to the filter. */ + outStream = new ostream( outFilter ); + } + else { + /* Writing out ot std out. */ + outStream = &cout; + } +} + +void InputData::openOutput() +{ + if ( outFilter != 0 ) { + outFilter->open( outputFileName, ios::out|ios::trunc ); + if ( !outFilter->is_open() ) { + error() << "error opening " << outputFileName << " for writing" << endl; + exit(1); + } + } +} + +void InputData::prepareMachineGen() +{ + if ( generateDot ) { + /* Locate a machine spec to generate dot output for. We can only emit. + * Dot takes one graph at a time. */ + if ( machineSpec != 0 ) { + /* Machine specified. */ + ParserDictEl *pdEl = parserDict.find( machineSpec ); + if ( pdEl == 0 ) + error() << "could not locate machine specified with -S and/or -M" << endp; + dotGenParser = pdEl->value; + } + else { + /* No machine spec given, just use the first one. */ + if ( parserList.length() == 0 ) + error() << "no machine specification to generate graphviz output" << endp; + + dotGenParser = parserList.head; + } + + GraphDictEl *gdEl = 0; + + if ( machineName != 0 ) { + gdEl = dotGenParser->pd->graphDict.find( machineName ); + if ( gdEl == 0 ) + error() << "machine definition/instantiation not found" << endp; + } + else { + /* We are using the whole machine spec. Need to make sure there + * are instances in the spec. */ + if ( dotGenParser->pd->instanceList.length() == 0 ) + error() << "no machine instantiations to generate graphviz output" << endp; + } + + dotGenParser->pd->prepareMachineGen( gdEl ); + } + else { + /* No machine spec or machine name given. Generate everything. */ + for ( ParserDict::Iter parser = parserDict; parser.lte(); parser++ ) { + ParseData *pd = parser->value->pd; + if ( pd->instanceList.length() > 0 ) + pd->prepareMachineGen( 0 ); + } + } +} + +void InputData::generateReduced() +{ + if ( generateDot ) + dotGenParser->pd->generateReduced( *this ); + else { + for ( ParserDict::Iter parser = parserDict; parser.lte(); parser++ ) { + ParseData *pd = parser->value->pd; + if ( pd->instanceList.length() > 0 ) + pd->generateReduced( *this ); + } + } +} + +/* Send eof to all parsers. */ +void InputData::terminateAllParsers( ) +{ + /* FIXME: a proper token is needed here. Suppose we should use the + * location of EOF in the last file that the parser was referenced in. */ + InputLoc loc; + loc.fileName = "<EOF>"; + loc.line = 0; + loc.col = 0; + for ( ParserDict::Iter pdel = parserDict; pdel.lte(); pdel++ ) + pdel->value->token( loc, Parser_tk_eof, 0, 0 ); +} + +void InputData::verifyWritesHaveData() +{ + if ( !generateXML && !generateDot ) { + for ( InputItemList::Iter ii = inputItems; ii.lte(); ii++ ) { + if ( ii->type == InputItem::Write ) { + if ( ii->pd->cgd == 0 ) + error( ii->loc ) << "no machine instantiations to write" << endl; + } + } + } +} + +void InputData::writeOutput() +{ + if ( generateXML ) + writeXML( *outStream ); + else if ( generateDot ) + static_cast<GraphvizDotGen*>(dotGenParser->pd->cgd)->writeDotFile(); + else { + bool hostLineDirective = true; + for ( InputItemList::Iter ii = inputItems; ii.lte(); ii++ ) { + if ( ii->type == InputItem::Write ) { + CodeGenData *cgd = ii->pd->cgd; + ::keyOps = &cgd->thisKeyOps; + + hostLineDirective = cgd->writeStatement( ii->loc, + ii->writeArgs.length()-1, ii->writeArgs.data ); + } + else { + if ( hostLineDirective ) { + /* Write statements can turn off host line directives for + * host sections that follow them. */ + *outStream << '\n'; + lineDirective( *outStream, inputFileName, ii->loc.line ); + } + *outStream << ii->data.str(); + hostLineDirective = true; + } + } + } +} + diff --git a/contrib/tools/ragel6/inputdata.h b/contrib/tools/ragel6/inputdata.h new file mode 100644 index 0000000000..09a62c15ac --- /dev/null +++ b/contrib/tools/ragel6/inputdata.h @@ -0,0 +1,106 @@ +/* + * Copyright 2008 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _INPUT_DATA +#define _INPUT_DATA + +#include "gendata.h" +#include <iostream> +#include <sstream> + +struct Parser; +struct ParseData; + +struct InputItem +{ + enum Type { + HostData, + Write, + }; + + Type type; + std::ostringstream data; + std::string name; + ParseData *pd; + Vector<char *> writeArgs; + + InputLoc loc; + + InputItem *prev, *next; +}; + +struct Parser; + +typedef AvlMap<const char*, Parser*, CmpStr> ParserDict; +typedef AvlMapEl<const char*, Parser*> ParserDictEl; +typedef DList<Parser> ParserList; +typedef DList<InputItem> InputItemList; +typedef Vector<const char *> ArgsVector; + +struct InputData +{ + InputData() : + inputFileName(0), + outputFileName(0), + inStream(0), + outStream(0), + outFilter(0), + dotGenParser(0) + {} + + /* The name of the root section, this does not change during an include. */ + const char *inputFileName; + const char *outputFileName; + + /* Io globals. */ + std::istream *inStream; + std::ostream *outStream; + output_filter *outFilter; + + Parser *dotGenParser; + + ParserDict parserDict; + ParserList parserList; + InputItemList inputItems; + + ArgsVector includePaths; + + void verifyWritesHaveData(); + + void writeOutput(); + void makeOutputStream(); + void openOutput(); + void generateReduced(); + void prepareMachineGen(); + void terminateAllParsers(); + + void cdDefaultFileName( const char *inputFile ); + void goDefaultFileName( const char *inputFile ); + void javaDefaultFileName( const char *inputFile ); + void rubyDefaultFileName( const char *inputFile ); + void csharpDefaultFileName( const char *inputFile ); + void ocamlDefaultFileName( const char *inputFile ); + + void writeLanguage( std::ostream &out ); + void writeXML( std::ostream &out ); +}; + +#endif diff --git a/contrib/tools/ragel6/javacodegen.cpp b/contrib/tools/ragel6/javacodegen.cpp new file mode 100644 index 0000000000..99fbbbcf4b --- /dev/null +++ b/contrib/tools/ragel6/javacodegen.cpp @@ -0,0 +1,1688 @@ +/* + * Copyright 2006-2007 Adrian Thurston <thurston@complang.org> + * 2007 Colin Fleming <colin.fleming@caverock.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "javacodegen.h" +#include "redfsm.h" +#include "gendata.h" +#include <iomanip> +#include <sstream> + +/* Integer array line length. */ +#define IALL 12 + +/* Static array initialization item count + * (should be multiple of IALL). */ +#define SAIIC 8184 + +#define _resume 1 +#define _again 2 +#define _eof_trans 3 +#define _test_eof 4 +#define _out 5 + +using std::setw; +using std::ios; +using std::ostringstream; +using std::string; +using std::cerr; + +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; +using std::setiosflags; + +void javaLineDirective( ostream &out, const char *fileName, int line ) +{ + /* Write the preprocessor line info for to the input file. */ + out << "// line " << line << " \""; + for ( const char *pc = fileName; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + out << "\\\\"; + else + out << *pc; + } + out << "\"\n"; +} + +void JavaTabCodeGen::genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + javaLineDirective( out, filter->fileName, filter->line + 1 ); +} + +void JavaTabCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "{" << vCS() << " = " << gotoDest << "; _goto_targ = " << _again << "; " << + CTRL_FLOW() << "continue _goto;}"; +} + +void JavaTabCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "{" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); _goto_targ = " << _again << "; " << CTRL_FLOW() << "continue _goto;}"; +} + +void JavaTabCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; _goto_targ = " << _again << "; " << CTRL_FLOW() << "continue _goto;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void JavaTabCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "{" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); _goto_targ = " << _again << "; " << CTRL_FLOW() << "continue _goto;}"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void JavaTabCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "{" << vCS() << " = " << STACK() << "[--" << TOP() << "];"; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + ret << "_goto_targ = " << _again << "; " << CTRL_FLOW() << "continue _goto;}"; +} + +void JavaTabCodeGen::BREAK( ostream &ret, int targState ) +{ + ret << "{ " << P() << " += 1; _goto_targ = " << _out << "; " << + CTRL_FLOW() << " continue _goto;}"; +} + +void JavaTabCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void JavaTabCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void JavaTabCodeGen::EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ) +{ + /* The parser gives fexec two children. The double brackets are for D + * code. If the inline list is a single word it will get interpreted as a + * C-style cast by the D compiler. */ + ret << "{" << P() << " = (("; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << "))-1;}"; +} + +/* Write out an inline tree structure. Walks the list and possibly calls out + * to virtual functions than handle language specific items in the tree. */ +void JavaTabCodeGen::INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Text: + ret << item->data; + break; + case GenInlineItem::Goto: + GOTO( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Call: + CALL( ret, item->targState->id, targState, inFinish ); + break; + case GenInlineItem::Next: + NEXT( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Ret: + RET( ret, inFinish ); + break; + case GenInlineItem::PChar: + ret << P(); + break; + case GenInlineItem::Char: + ret << GET_KEY(); + break; + case GenInlineItem::Hold: + ret << P() << "--;"; + break; + case GenInlineItem::Exec: + EXEC( ret, item, targState, inFinish ); + break; + case GenInlineItem::Curs: + ret << "(_ps)"; + break; + case GenInlineItem::Targs: + ret << "(" << vCS() << ")"; + break; + case GenInlineItem::Entry: + ret << item->targState->id; + break; + case GenInlineItem::GotoExpr: + GOTO_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::CallExpr: + CALL_EXPR( ret, item, targState, inFinish ); + break; + case GenInlineItem::NextExpr: + NEXT_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::LmSwitch: + LM_SWITCH( ret, item, targState, inFinish ); + break; + case GenInlineItem::LmSetActId: + SET_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokEnd: + SET_TOKEND( ret, item ); + break; + case GenInlineItem::LmGetTokEnd: + GET_TOKEND( ret, item ); + break; + case GenInlineItem::LmInitTokStart: + INIT_TOKSTART( ret, item ); + break; + case GenInlineItem::LmInitAct: + INIT_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokStart: + SET_TOKSTART( ret, item ); + break; + case GenInlineItem::SubAction: + SUB_ACTION( ret, item, targState, inFinish ); + break; + case GenInlineItem::Break: + BREAK( ret, targState ); + break; + } + } +} + +string JavaTabCodeGen::DATA_PREFIX() +{ + if ( !noPrefix ) + return FSM_NAME() + "_"; + return ""; +} + +/* Emit the alphabet data type. */ +string JavaTabCodeGen::ALPH_TYPE() +{ + string ret = keyOps->alphType->data1; + if ( keyOps->alphType->data2 != 0 ) { + ret += " "; + ret += + keyOps->alphType->data2; + } + return ret; +} + +/* Emit the alphabet data type. */ +string JavaTabCodeGen::WIDE_ALPH_TYPE() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + ret = ALPH_TYPE(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + assert( wideType != 0 ); + + ret = wideType->data1; + if ( wideType->data2 != 0 ) { + ret += " "; + ret += wideType->data2; + } + } + return ret; +} + + + +void JavaTabCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n" + " _keys = " << CO() << "[" << vCS() << "]*2\n;" + " _klen = " << CL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " int _lower = _keys\n;" + " int _mid;\n" + " int _upper = _keys + (_klen<<1) - 2;\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1);\n" + " if ( " << GET_WIDE_KEY() << " < " << CK() << "[_mid] )\n" + " _upper = _mid - 2;\n" + " else if ( " << GET_WIDE_KEY() << " > " << CK() << "[_mid+1] )\n" + " _lower = _mid + 2;\n" + " else {\n" + " switch ( " << C() << "[" << CO() << "[" << vCS() << "]" + " + ((_mid - _keys)>>1)] ) {\n" + ; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId << ": {\n"; + out << TABS(2) << "_widec = " << KEY(condSpace->baseKey) << + " + (" << GET_KEY() << " - " << KEY(keyOps->minKey) << ");\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << + " break;\n" + " }\n"; + } + + out << + " }\n" + " break;\n" + " }\n" + " }\n" + " }\n" + "\n"; +} + + +void JavaTabCodeGen::LOCATE_TRANS() +{ + out << + " _match: do {\n" + " _keys = " << KO() << "[" << vCS() << "]" << ";\n" + " _trans = " << IO() << "[" << vCS() << "];\n" + " _klen = " << SL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " int _lower = _keys;\n" + " int _mid;\n" + " int _upper = _keys + _klen - 1;\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + ((_upper-_lower) >> 1);\n" + " if ( " << GET_WIDE_KEY() << " < " << K() << "[_mid] )\n" + " _upper = _mid - 1;\n" + " else if ( " << GET_WIDE_KEY() << " > " << K() << "[_mid] )\n" + " _lower = _mid + 1;\n" + " else {\n" + " _trans += (_mid - _keys);\n" + " break _match;\n" + " }\n" + " }\n" + " _keys += _klen;\n" + " _trans += _klen;\n" + " }\n" + "\n" + " _klen = " << RL() << "[" << vCS() << "];\n" + " if ( _klen > 0 ) {\n" + " int _lower = _keys;\n" + " int _mid;\n" + " int _upper = _keys + (_klen<<1) - 2;\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1);\n" + " if ( " << GET_WIDE_KEY() << " < " << K() << "[_mid] )\n" + " _upper = _mid - 2;\n" + " else if ( " << GET_WIDE_KEY() << " > " << K() << "[_mid+1] )\n" + " _lower = _mid + 2;\n" + " else {\n" + " _trans += ((_mid - _keys)>>1);\n" + " break _match;\n" + " }\n" + " }\n" + " _trans += _klen;\n" + " }\n" + " } while (false);\n" + "\n"; +} + +/* Determine if we should use indicies or not. */ +void JavaTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActionLoc) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActionLoc) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +int JavaTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +int JavaTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +int JavaTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + + +int JavaTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + return act; +} + +std::ostream &JavaTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &JavaTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &JavaTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, true ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &JavaTabCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\tcase " << act->actionId << ":\n"; + ACTION( out, act, 0, false ); + out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &JavaTabCodeGen::COND_OFFSETS() +{ + int curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + ARRAY_ITEM( INT(curKeyOffset), st.last() ); + + /* Move the key offset ahead. */ + curKeyOffset += st->stateCondList.length(); + } + return out; +} + +std::ostream &JavaTabCodeGen::KEY_OFFSETS() +{ + int curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + ARRAY_ITEM( INT(curKeyOffset), st.last() ); + + /* Move the key offset ahead. */ + curKeyOffset += st->outSingle.length() + st->outRange.length()*2; + } + return out; +} + + +std::ostream &JavaTabCodeGen::INDEX_OFFSETS() +{ + int curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + ARRAY_ITEM( INT(curIndOffset), st.last() ); + + /* Move the index offset ahead. */ + curIndOffset += st->outSingle.length() + st->outRange.length(); + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + return out; +} + +std::ostream &JavaTabCodeGen::COND_LENS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + ARRAY_ITEM( INT(st->stateCondList.length()), st.last() ); + } + return out; +} + + +std::ostream &JavaTabCodeGen::SINGLE_LENS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + ARRAY_ITEM( INT(st->outSingle.length()), st.last() ); + } + return out; +} + +std::ostream &JavaTabCodeGen::RANGE_LENS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit length of range index. */ + ARRAY_ITEM( INT(st->outRange.length()), st.last() ); + } + return out; +} + +std::ostream &JavaTabCodeGen::TO_STATE_ACTIONS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(TO_STATE_ACTION(st)), st.last() ); + } + return out; +} + +std::ostream &JavaTabCodeGen::FROM_STATE_ACTIONS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(FROM_STATE_ACTION(st)), st.last() ); + } + return out; +} + +std::ostream &JavaTabCodeGen::EOF_ACTIONS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(EOF_ACTION(st)), st.last() ); + } + return out; +} + +std::ostream &JavaTabCodeGen::EOF_TRANS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + + /* Write any eof action. */ + ARRAY_ITEM( INT(trans), st.last() ); + } + return out; +} + + +std::ostream &JavaTabCodeGen::COND_KEYS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Lower key. */ + ARRAY_ITEM( KEY( sc->lowKey ), false ); + ARRAY_ITEM( KEY( sc->highKey ), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + +std::ostream &JavaTabCodeGen::COND_SPACES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Cond Space id. */ + ARRAY_ITEM( KEY( sc->condSpace->condSpaceId ), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + +std::ostream &JavaTabCodeGen::KEYS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + ARRAY_ITEM( KEY( stel->lowKey ), false ); + } + + /* Loop the state's transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + /* Lower key. */ + ARRAY_ITEM( KEY( rtel->lowKey ), false ); + + /* Upper key. */ + ARRAY_ITEM( KEY( rtel->highKey ), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + +std::ostream &JavaTabCodeGen::INDICIES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + ARRAY_ITEM( KEY( stel->value->id ), false ); + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + ARRAY_ITEM( KEY( rtel->value->id ), false ); + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + ARRAY_ITEM( KEY( st->defTrans->id ), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + +std::ostream &JavaTabCodeGen::TRANS_TARGS() +{ + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + ARRAY_ITEM( KEY( trans->targ->id ), false ); + totalTrans++; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + ARRAY_ITEM( KEY( trans->targ->id ), false ); + totalTrans++; + } + + /* The state's default target state. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + ARRAY_ITEM( KEY( trans->targ->id ), false ); + totalTrans++; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + trans->pos = totalTrans++; + ARRAY_ITEM( KEY( trans->targ->id ), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + + +std::ostream &JavaTabCodeGen::TRANS_ACTIONS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), false ); + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), false ); + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), false ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), true ); + return out; +} + +std::ostream &JavaTabCodeGen::TRANS_TARGS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Save the position. Needed for eofTargs. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + ARRAY_ITEM( INT(trans->targ->id), ( t >= redFsm->transSet.length()-1 ) ); + } + delete[] transPtrs; + return out; +} + + +std::ostream &JavaTabCodeGen::TRANS_ACTIONS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ( t >= redFsm->transSet.length()-1 ) ); + } + delete[] transPtrs; + return out; +} + +void JavaTabCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + STATIC_VAR( ALPH_TYPE(), DATA_PREFIX() + "ex_" + ex->name ) + << " = " << KEY(ex->key) << ";\n"; + } + out << "\n"; + } +} + +void JavaTabCodeGen::writeStart() +{ + out << START_STATE_ID(); +} + +void JavaTabCodeGen::writeFirstFinal() +{ + out << FIRST_FINAL_STATE(); +} + +void JavaTabCodeGen::writeError() +{ + out << ERROR_STATE(); +} + +void JavaTabCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->startState != 0 ) + STATIC_VAR( "int", START() ) << " = " << START_STATE_ID() << ";\n"; + + if ( !noFinal ) + STATIC_VAR( "int" , FIRST_FINAL() ) << " = " << FIRST_FINAL_STATE() << ";\n"; + + if ( !noError ) + STATIC_VAR( "int", ERROR() ) << " = " << ERROR_STATE() << ";\n"; + + out << "\n"; + + if ( !noEntry && entryPointNames.length() > 0 ) { + for ( EntryNameVect::Iter en = entryPointNames; en.lte(); en++ ) { + STATIC_VAR( "int", DATA_PREFIX() + "en_" + *en ) << + " = " << entryPointIds[en.pos()] << ";\n"; + } + out << "\n"; + } +} + +void JavaTabCodeGen::writeExec() +{ + out << + " {\n" + " int _klen"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + out << + ";\n" + " int _trans = 0;\n"; + + if ( redFsm->anyConditions() ) + out << " int _widec;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() || + redFsm->anyFromStateActions() ) + { + out << + " int _acts;\n" + " int _nacts;\n"; + } + + out << + " int _keys;\n" + " int _goto_targ = 0;\n" + "\n"; + + out << + " _goto: while (true) {\n" + " switch ( _goto_targ ) {\n" + " case 0:\n"; + + if ( !noEnd ) { + out << + " if ( " << P() << " == " << PE() << " ) {\n" + " _goto_targ = " << _test_eof << ";\n" + " continue _goto;\n" + " }\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " ) {\n" + " _goto_targ = " << _out << ";\n" + " continue _goto;\n" + " }\n"; + } + + out << "case " << _resume << ":\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" << vCS() << "]" << ";\n" + " _nacts = " << CAST("int") << " " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + FROM_STATE_ACTION_SWITCH() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( useIndicies ) + out << " _trans = " << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) + out << "case " << _eof_trans << ":\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if ( " << TA() << "[_trans] != 0 ) {\n" + " _acts = " << TA() << "[_trans]" << ";\n" + " _nacts = " << CAST("int") << " " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 )\n {\n" + " switch ( " << A() << "[_acts++] )\n" + " {\n"; + ACTION_SWITCH() << + " }\n" + " }\n" + " }\n" + "\n"; + } + + out << "case " << _again << ":\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "]" << ";\n" + " _nacts = " << CAST("int") << " " << A() << "[_acts++];\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( " << A() << "[_acts++] ) {\n"; + TO_STATE_ACTION_SWITCH() << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " ) {\n" + " _goto_targ = " << _out << ";\n" + " continue _goto;\n" + " }\n"; + } + + if ( !noEnd ) { + out << + " if ( ++" << P() << " != " << PE() << " ) {\n" + " _goto_targ = " << _resume << ";\n" + " continue _goto;\n" + " }\n"; + } + else { + out << + " " << P() << " += 1;\n" + " _goto_targ = " << _resume << ";\n" + " continue _goto;\n"; + } + + out << "case " << _test_eof << ":\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if ( " << P() << " == " << vEOF() << " )\n" + " {\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if ( " << ET() << "[" << vCS() << "] > 0 ) {\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " _goto_targ = " << _eof_trans << ";\n" + " continue _goto;\n" + " }\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " int __acts = " << EA() << "[" << vCS() << "]" << ";\n" + " int __nacts = " << CAST("int") << " " << A() << "[__acts++];\n" + " while ( __nacts-- > 0 ) {\n" + " switch ( " << A() << "[__acts++] ) {\n"; + EOF_ACTION_SWITCH() << + " }\n" + " }\n"; + } + + out << + " }\n" + "\n"; + } + + out << "case " << _out << ":\n"; + + /* The switch and goto loop. */ + out << " }\n"; + out << " break; }\n"; + + /* The execute block. */ + out << " }\n"; +} + +std::ostream &JavaTabCodeGen::OPEN_ARRAY( string type, string name ) +{ + array_type = type; + array_name = name; + item_count = 0; + div_count = 1; + + out << "private static " << type << "[] init_" << name << "_0()\n" + "{\n\t" + "return new " << type << " [] {\n\t"; + return out; +} + +std::ostream &JavaTabCodeGen::ARRAY_ITEM( string item, bool last ) +{ + item_count++; + + out << setw(5) << setiosflags(ios::right) << item; + + if ( !last ) { + if ( item_count % SAIIC == 0 ) { + out << "\n\t};\n};\n" + "private static "<< array_type << "[] init_" << + array_name << "_" << div_count << "()\n" + "{\n\t" + "return new " << array_type << " [] {\n\t"; + div_count++; + } else if (item_count % IALL == 0) { + out << ",\n\t"; + } else { + out << ","; + } + } + return out; +} + +std::ostream &JavaTabCodeGen::CLOSE_ARRAY() +{ + out << "\n\t};\n}\n\n"; + + if (item_count < SAIIC) { + out << "private static final " << array_type << " " << array_name << + "[] = init_" << array_name << "_0();\n\n"; + } else { + out << "private static final " << array_type << " [] combine_" << array_name + << "() {\n\t" + << array_type << " [] combined = new " << array_type << + " [ " << item_count << " ];\n\t"; + int block = 0; + int full_blocks = item_count / SAIIC; + for (;block < full_blocks; ++block) { + out << "System.arraycopy ( init_" << array_name << "_" << block << + "(), 0, combined, " << SAIIC * block << ", " << SAIIC << " );\n\t"; + } + if ( (item_count % SAIIC) > 0 ) { + out << "System.arraycopy ( init_" << array_name << "_" << block << + "(), 0, combined, " << SAIIC * block << ", " << + (item_count % SAIIC) << " );\n\t"; + } + out << "return combined;\n}\n"; + out << "private static final " << array_type << " [] " << array_name << + " = combine_" << array_name << "();"; + } + return out; +} + + +std::ostream &JavaTabCodeGen::STATIC_VAR( string type, string name ) +{ + out << "static final " << type << " " << name; + return out; +} + +string JavaTabCodeGen::ARR_OFF( string ptr, string offset ) +{ + return ptr + " + " + offset; +} + +string JavaTabCodeGen::CAST( string type ) +{ + return "(" + type + ")"; +} + +string JavaTabCodeGen::NULL_ITEM() +{ + /* In java we use integers instead of pointers. */ + return "-1"; +} + +string JavaTabCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + ret << DATA() << "[" << P() << "]"; + } + return ret.str(); +} + +string JavaTabCodeGen::CTRL_FLOW() +{ + return "if (true) "; +} + +unsigned int JavaTabCodeGen::arrayTypeSize( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + return arrayType->size; +} + +string JavaTabCodeGen::ARRAY_TYPE( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + + string ret = arrayType->data1; + if ( arrayType->data2 != 0 ) { + ret += " "; + ret += arrayType->data2; + } + return ret; +} + + +/* Write out the fsm name. */ +string JavaTabCodeGen::FSM_NAME() +{ + return fsmName; +} + +/* Emit the offset of the start state as a decimal integer. */ +string JavaTabCodeGen::START_STATE_ID() +{ + ostringstream ret; + ret << redFsm->startState->id; + return ret.str(); +}; + +/* Write out the array of actions. */ +std::ostream &JavaTabCodeGen::ACTIONS_ARRAY() +{ + ARRAY_ITEM( INT(0), false ); + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + /* Write out the length, which will never be the last character. */ + ARRAY_ITEM( INT(act->key.length()), false ); + + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) + ARRAY_ITEM( INT(item->value->actionId), (act.last() && item.last()) ); + } + return out; +} + + +string JavaTabCodeGen::ACCESS() +{ + ostringstream ret; + if ( accessExpr != 0 ) + INLINE_LIST( ret, accessExpr, 0, false ); + return ret.str(); +} + +string JavaTabCodeGen::P() +{ + ostringstream ret; + if ( pExpr == 0 ) + ret << "p"; + else { + ret << "("; + INLINE_LIST( ret, pExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::PE() +{ + ostringstream ret; + if ( peExpr == 0 ) + ret << "pe"; + else { + ret << "("; + INLINE_LIST( ret, peExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::vEOF() +{ + ostringstream ret; + if ( eofExpr == 0 ) + ret << "eof"; + else { + ret << "("; + INLINE_LIST( ret, eofExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::vCS() +{ + ostringstream ret; + if ( csExpr == 0 ) + ret << ACCESS() << "cs"; + else { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, csExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::TOP() +{ + ostringstream ret; + if ( topExpr == 0 ) + ret << ACCESS() + "top"; + else { + ret << "("; + INLINE_LIST( ret, topExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::STACK() +{ + ostringstream ret; + if ( stackExpr == 0 ) + ret << ACCESS() + "stack"; + else { + ret << "("; + INLINE_LIST( ret, stackExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::ACT() +{ + ostringstream ret; + if ( actExpr == 0 ) + ret << ACCESS() + "act"; + else { + ret << "("; + INLINE_LIST( ret, actExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::TOKSTART() +{ + ostringstream ret; + if ( tokstartExpr == 0 ) + ret << ACCESS() + "ts"; + else { + ret << "("; + INLINE_LIST( ret, tokstartExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::TOKEND() +{ + ostringstream ret; + if ( tokendExpr == 0 ) + ret << ACCESS() + "te"; + else { + ret << "("; + INLINE_LIST( ret, tokendExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string JavaTabCodeGen::DATA() +{ + ostringstream ret; + if ( dataExpr == 0 ) + ret << ACCESS() + "data"; + else { + ret << "("; + INLINE_LIST( ret, dataExpr, 0, false ); + ret << ")"; + } + return ret.str(); +} + + +string JavaTabCodeGen::GET_WIDE_KEY() +{ + if ( redFsm->anyConditions() ) + return "_widec"; + else + return GET_KEY(); +} + +string JavaTabCodeGen::GET_WIDE_KEY( RedStateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) + return "_widec"; + else + return GET_KEY(); +} + +/* Write out level number of tabs. Makes the nested binary search nice + * looking. */ +string JavaTabCodeGen::TABS( int level ) +{ + string result; + while ( level-- > 0 ) + result += "\t"; + return result; +} + +string JavaTabCodeGen::KEY( Key key ) +{ + ostringstream ret; + if ( keyOps->isSigned || !hostLang->explicitUnsigned ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal(); + return ret.str(); +} + +string JavaTabCodeGen::INT( int i ) +{ + ostringstream ret; + ret << i; + return ret.str(); +} + +void JavaTabCodeGen::LM_SWITCH( ostream &ret, GenInlineItem *item, + int targState, int inFinish ) +{ + ret << + " switch( " << ACT() << " ) {\n"; + + for ( GenInlineList::Iter lma = *item->children; lma.lte(); lma++ ) { + /* Write the case label, the action and the case break. */ + if ( lma->lmId < 0 ) + ret << " default:\n"; + else + ret << " case " << lma->lmId << ":\n"; + + /* Write the block and close it off. */ + ret << " {"; + INLINE_LIST( ret, lma->children, targState, inFinish ); + ret << "}\n"; + + ret << " break;\n"; + } + + ret << + " }\n" + "\t"; +} + +void JavaTabCodeGen::SET_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = " << item->lmId << ";"; +} + +void JavaTabCodeGen::SET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + /* The tokend action sets tokend. */ + ret << TOKEND() << " = " << P(); + if ( item->offset != 0 ) + out << "+" << item->offset; + out << ";"; +} + +void JavaTabCodeGen::GET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + ret << TOKEND(); +} + +void JavaTabCodeGen::INIT_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << NULL_ITEM() << ";"; +} + +void JavaTabCodeGen::INIT_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = 0;"; +} + +void JavaTabCodeGen::SET_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << P() << ";"; +} + +void JavaTabCodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << "{"; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << "}"; + } +} + +void JavaTabCodeGen::ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ) +{ + /* Write the preprocessor line info for going into the source file. */ + javaLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + ret << "\t{"; + INLINE_LIST( ret, action->inlineList, targState, inFinish ); + ret << "}\n"; +} + +void JavaTabCodeGen::CONDITION( ostream &ret, GenAction *condition ) +{ + ret << "\n"; + javaLineDirective( ret, condition->loc.fileName, condition->loc.line ); + INLINE_LIST( ret, condition->inlineList, 0, false ); +} + +string JavaTabCodeGen::ERROR_STATE() +{ + ostringstream ret; + if ( redFsm->errState != 0 ) + ret << redFsm->errState->id; + else + ret << "-1"; + return ret.str(); +} + +string JavaTabCodeGen::FIRST_FINAL_STATE() +{ + ostringstream ret; + if ( redFsm->firstFinState != 0 ) + ret << redFsm->firstFinState->id; + else + ret << redFsm->nextStateId; + return ret.str(); +} + +void JavaTabCodeGen::writeInit() +{ + out << " {\n"; + + if ( !noCS ) + out << "\t" << vCS() << " = " << START() << ";\n"; + + /* If there are any calls, then the stack top needs initialization. */ + if ( redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\t" << TOP() << " = 0;\n"; + + if ( hasLongestMatch ) { + out << + " " << TOKSTART() << " = " << NULL_ITEM() << ";\n" + " " << TOKEND() << " = " << NULL_ITEM() << ";\n" + " " << ACT() << " = 0;\n"; + } + out << " }\n"; +} + +void JavaTabCodeGen::finishRagelDef() +{ + /* The frontend will do this for us, but it may be a good idea to force it + * if the intermediate file is edited. */ + redFsm->sortByStateId(); + + /* Choose default transitions and the single transition. */ + redFsm->chooseDefaultSpan(); + + /* Maybe do flat expand, otherwise choose single. */ + redFsm->chooseSingle(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + /* Anlayze Machine will find the final action reference counts, among + * other things. We will use these in reporting the usage + * of fsm directives in action code. */ + analyzeMachine(); + + /* Determine if we should use indicies. */ + calcIndexSize(); +} + +ostream &JavaTabCodeGen::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &JavaTabCodeGen::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + +#undef _resume +#undef _again +#undef _eof_trans +#undef _test_eof +#undef _out diff --git a/contrib/tools/ragel6/javacodegen.h b/contrib/tools/ragel6/javacodegen.h new file mode 100644 index 0000000000..06914c9c52 --- /dev/null +++ b/contrib/tools/ragel6/javacodegen.h @@ -0,0 +1,197 @@ +/* + * Copyright 2006-2007 Adrian Thurston <thurston@complang.org> + * 2007 Colin Fleming <colin.fleming@caverock.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _JAVACODEGEN_H +#define _JAVACODEGEN_H + +#include <iostream> +#include <string> +#include <stdio.h> +#include "common.h" +#include "gendata.h" + +using std::string; +using std::ostream; + +/* + * JavaTabCodeGen + */ +struct JavaTabCodeGen : public CodeGenData +{ + JavaTabCodeGen( ostream &out ) : + CodeGenData(out) {} + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + std::ostream &COND_KEYS(); + std::ostream &COND_SPACES(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &COND_OFFSETS(); + std::ostream &KEY_OFFSETS(); + std::ostream &INDEX_OFFSETS(); + std::ostream &COND_LENS(); + std::ostream &SINGLE_LENS(); + std::ostream &RANGE_LENS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + std::ostream &TRANS_TARGS_WI(); + std::ostream &TRANS_ACTIONS_WI(); + + void BREAK( ostream &ret, int targState ); + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &ret, bool inFinish ); + + void COND_TRANSLATE(); + void LOCATE_TRANS(); + + virtual void writeExec(); + virtual void writeData(); + virtual void writeInit(); + virtual void writeExports(); + virtual void writeStart(); + virtual void writeFirstFinal(); + virtual void writeError(); + virtual void finishRagelDef(); + + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + + int TO_STATE_ACTION( RedStateAp *state ); + int FROM_STATE_ACTION( RedStateAp *state ); + int EOF_ACTION( RedStateAp *state ); + int TRANS_ACTION( RedTransAp *trans ); + + /* Determine if we should use indicies. */ + void calcIndexSize(); + +private: + string array_type; + string array_name; + int item_count; + int div_count; + +public: + + virtual string NULL_ITEM(); + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &ARRAY_ITEM( string item, bool last ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + virtual string ARR_OFF( string ptr, string offset ); + virtual string CAST( string type ); + virtual string GET_KEY(); + virtual string CTRL_FLOW(); + + string FSM_NAME(); + string START_STATE_ID(); + ostream &ACTIONS_ARRAY(); + string GET_WIDE_KEY(); + string GET_WIDE_KEY( RedStateAp *state ); + string TABS( int level ); + string KEY( Key key ); + string INT( int i ); + void ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ); + void CONDITION( ostream &ret, GenAction *condition ); + string ALPH_TYPE(); + string WIDE_ALPH_TYPE(); + string ARRAY_TYPE( unsigned long maxVal ); + + string ACCESS(); + + string P(); + string PE(); + string vEOF(); + + string vCS(); + string STACK(); + string TOP(); + string TOKSTART(); + string TOKEND(); + string ACT(); + string DATA(); + + string DATA_PREFIX(); + string PM() { return "_" + DATA_PREFIX() + "partition_map"; } + string C() { return "_" + DATA_PREFIX() + "cond_spaces"; } + string CK() { return "_" + DATA_PREFIX() + "cond_keys"; } + string K() { return "_" + DATA_PREFIX() + "trans_keys"; } + string I() { return "_" + DATA_PREFIX() + "indicies"; } + string CO() { return "_" + DATA_PREFIX() + "cond_offsets"; } + string KO() { return "_" + DATA_PREFIX() + "key_offsets"; } + string IO() { return "_" + DATA_PREFIX() + "index_offsets"; } + string CL() { return "_" + DATA_PREFIX() + "cond_lengths"; } + string SL() { return "_" + DATA_PREFIX() + "single_lengths"; } + string RL() { return "_" + DATA_PREFIX() + "range_lengths"; } + string A() { return "_" + DATA_PREFIX() + "actions"; } + string TA() { return "_" + DATA_PREFIX() + "trans_actions"; } + string TT() { return "_" + DATA_PREFIX() + "trans_targs"; } + string TSA() { return "_" + DATA_PREFIX() + "to_state_actions"; } + string FSA() { return "_" + DATA_PREFIX() + "from_state_actions"; } + string EA() { return "_" + DATA_PREFIX() + "eof_actions"; } + string ET() { return "_" + DATA_PREFIX() + "eof_trans"; } + string SP() { return "_" + DATA_PREFIX() + "key_spans"; } + string CSP() { return "_" + DATA_PREFIX() + "cond_key_spans"; } + string START() { return DATA_PREFIX() + "start"; } + string ERROR() { return DATA_PREFIX() + "error"; } + string FIRST_FINAL() { return DATA_PREFIX() + "first_final"; } + string CTXDATA() { return DATA_PREFIX() + "ctxdata"; } + + void INLINE_LIST( ostream &ret, GenInlineList *inlineList, int targState, bool inFinish ); + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void EXECTE( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ); + + string ERROR_STATE(); + string FIRST_FINAL_STATE(); + + ostream &source_warning(const InputLoc &loc); + ostream &source_error(const InputLoc &loc); + + unsigned int arrayTypeSize( unsigned long maxVal ); + + bool outLabelUsed; + bool againLabelUsed; + bool useIndicies; + + void genLineDirective( ostream &out ); +}; + +#endif diff --git a/contrib/tools/ragel6/main.cpp b/contrib/tools/ragel6/main.cpp new file mode 100644 index 0000000000..1fd36c6f9d --- /dev/null +++ b/contrib/tools/ragel6/main.cpp @@ -0,0 +1,575 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <stdlib.h> +#include <string.h> +#include <stdio.h> +#include <iostream> +#include <fstream> +#ifndef _WIN32 +# include <unistd.h> +#endif +#include <sstream> +#include <sys/types.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <errno.h> + +#ifdef _WIN32 +#include <windows.h> +#include <psapi.h> +#include <time.h> +#include <io.h> +#include <process.h> + +#if _MSC_VER +#define S_IRUSR _S_IREAD +#define S_IWUSR _S_IWRITE +#endif +#endif + +/* Parsing. */ +#include "ragel.h" +#include "rlscan.h" + +/* Parameters and output. */ +#include "pcheck.h" +#include "vector.h" +#include "version.h" +#include "common.h" +#include "inputdata.h" + +using std::istream; +using std::ostream; +using std::ifstream; +using std::ofstream; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; +using std::ios; +using std::streamsize; + +/* Controls minimization. */ +MinimizeLevel minimizeLevel = MinimizePartition2; +MinimizeOpt minimizeOpt = MinimizeMostOps; + +/* Graphviz dot file generation. */ +const char *machineSpec = 0, *machineName = 0; +bool machineSpecFound = false; +bool wantDupsRemoved = true; + +bool printStatistics = false; +bool generateXML = false; +bool generateDot = false; + +/* Target language and output style. */ +CodeStyle codeStyle = GenTables; + +int numSplitPartitions = 0; +bool noLineDirectives = false; + +bool displayPrintables = false; + +/* Target ruby impl */ +RubyImplEnum rubyImpl = MRI; + +/* Print a summary of the options. */ +void usage() +{ + cout << +"usage: ragel [options] file\n" +"general:\n" +" -h, -H, -?, --help Print this usage and exit\n" +" -v, --version Print version information and exit\n" +" -o <file> Write output to <file>\n" +" -s Print some statistics on stderr\n" +" -d Do not remove duplicates from action lists\n" +" -I <dir> Add <dir> to the list of directories to search\n" +" for included an imported files\n" +"error reporting format:\n" +" --error-format=gnu file:line:column: message (default)\n" +" --error-format=msvc file(line,column): message\n" +"fsm minimization:\n" +" -n Do not perform minimization\n" +" -m Minimize at the end of the compilation\n" +" -l Minimize after most operations (default)\n" +" -e Minimize after every operation\n" +"visualization:\n" +" -x Run the frontend only: emit XML intermediate format\n" +" -V Generate a dot file for Graphviz\n" +" -p Display printable characters on labels\n" +" -S <spec> FSM specification to output (for graphviz output)\n" +" -M <machine> Machine definition/instantiation to output (for graphviz output)\n" +"host language:\n" +" -C The host language is C, C++, Obj-C or Obj-C++ (default)\n" +" -D The host language is D\n" +" -Z The host language is Go\n" +" -J The host language is Java\n" +" -R The host language is Ruby\n" +" -A The host language is C#\n" +" -O The host language is OCaml\n" +"line directives: (C/D/Ruby/C#/OCaml)\n" +" -L Inhibit writing of #line directives\n" +"code style: (C/D/Java/Ruby/C#/OCaml)\n" +" -T0 Table driven FSM (default)\n" +"code style: (C/D/Ruby/C#/OCaml)\n" +" -T1 Faster table driven FSM\n" +" -F0 Flat table driven FSM\n" +" -F1 Faster flat table-driven FSM\n" +"code style: (C/D/C#/OCaml)\n" +" -G0 Goto-driven FSM\n" +" -G1 Faster goto-driven FSM\n" +"code style: (C/D)\n" +" -G2 Really fast goto-driven FSM\n" +" -P<N> N-Way Split really fast goto-driven FSM\n" + ; + + exit(0); +} + +/* Print version information and exit. */ +void version() +{ + cout << "Ragel State Machine Compiler version " VERSION << " " PUBDATE << endl << + "Copyright (c) 2001-2009 by Adrian Thurston" << endl; + exit(0); +} + +/* Error reporting format. */ +ErrorFormat errorFormat = ErrorFormatGNU; + +InputLoc makeInputLoc( const char *fileName, int line, int col) +{ + InputLoc loc = { fileName, line, col }; + return loc; +} + +ostream &operator<<( ostream &out, const InputLoc &loc ) +{ + assert( loc.fileName != 0 ); + switch ( errorFormat ) { + case ErrorFormatMSVC: + out << loc.fileName << "(" << loc.line; + if ( loc.col ) + out << "," << loc.col; + out << ")"; + break; + + default: + out << loc.fileName << ":" << loc.line; + if ( loc.col ) + out << ":" << loc.col; + break; + } + return out; +} + +/* Total error count. */ +int gblErrorCount = 0; + +/* Print the opening to a warning in the input, then return the error ostream. */ +ostream &warning( const InputLoc &loc ) +{ + cerr << loc << ": warning: "; + return cerr; +} + +/* Print the opening to a program error, then return the error stream. */ +ostream &error() +{ + gblErrorCount += 1; + cerr << PROGNAME ": "; + return cerr; +} + +ostream &error( const InputLoc &loc ) +{ + gblErrorCount += 1; + cerr << loc << ": "; + return cerr; +} + +void escapeLineDirectivePath( std::ostream &out, char *path ) +{ + for ( char *pc = path; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + out << "\\\\"; + else + out << *pc; + } +} + +void processArgs( int argc, const char **argv, InputData &id ) +{ + ParamCheck pc("xo:dnmleabjkS:M:I:CDEJZRAOvHh?-:sT:F:G:P:LpV", argc, argv); + + /* FIXME: Need to check code styles VS langauge. */ + + while ( pc.check() ) { + switch ( pc.state ) { + case ParamCheck::match: + switch ( pc.parameter ) { + case 'V': + generateDot = true; + break; + + case 'x': + generateXML = true; + break; + + /* Output. */ + case 'o': + if ( *pc.paramArg == 0 ) + error() << "a zero length output file name was given" << endl; + else if ( id.outputFileName != 0 ) + error() << "more than one output file name was given" << endl; + else { + /* Ok, remember the output file name. */ + id.outputFileName = pc.paramArg; + } + break; + + /* Flag for turning off duplicate action removal. */ + case 'd': + wantDupsRemoved = false; + break; + + /* Minimization, mostly hidden options. */ + case 'n': + minimizeOpt = MinimizeNone; + break; + case 'm': + minimizeOpt = MinimizeEnd; + break; + case 'l': + minimizeOpt = MinimizeMostOps; + break; + case 'e': + minimizeOpt = MinimizeEveryOp; + break; + case 'a': + minimizeLevel = MinimizeApprox; + break; + case 'b': + minimizeLevel = MinimizeStable; + break; + case 'j': + minimizeLevel = MinimizePartition1; + break; + case 'k': + minimizeLevel = MinimizePartition2; + break; + + /* Machine spec. */ + case 'S': + if ( *pc.paramArg == 0 ) + error() << "please specify an argument to -S" << endl; + else if ( machineSpec != 0 ) + error() << "more than one -S argument was given" << endl; + else { + /* Ok, remember the path to the machine to generate. */ + machineSpec = pc.paramArg; + } + break; + + /* Machine path. */ + case 'M': + if ( *pc.paramArg == 0 ) + error() << "please specify an argument to -M" << endl; + else if ( machineName != 0 ) + error() << "more than one -M argument was given" << endl; + else { + /* Ok, remember the machine name to generate. */ + machineName = pc.paramArg; + } + break; + + case 'I': + if ( *pc.paramArg == 0 ) + error() << "please specify an argument to -I" << endl; + else { + id.includePaths.append( pc.paramArg ); + } + break; + + /* Host language types. */ + case 'C': + hostLang = &hostLangC; + break; + case 'D': + hostLang = &hostLangD; + break; + case 'E': + hostLang = &hostLangD2; + break; + case 'Z': + hostLang = &hostLangGo; + break; + case 'J': + hostLang = &hostLangJava; + break; + case 'R': + hostLang = &hostLangRuby; + break; + case 'A': + hostLang = &hostLangCSharp; + break; + case 'O': + hostLang = &hostLangOCaml; + break; + + /* Version and help. */ + case 'v': + version(); + break; + case 'H': case 'h': case '?': + usage(); + break; + case 's': + printStatistics = true; + break; + case '-': { + char *arg = strdup( pc.paramArg ); + char *eq = strchr( arg, '=' ); + + if ( eq != 0 ) + *eq++ = 0; + + if ( strcmp( arg, "help" ) == 0 ) + usage(); + else if ( strcmp( arg, "version" ) == 0 ) + version(); + else if ( strcmp( arg, "error-format" ) == 0 ) { + if ( eq == 0 ) + error() << "expecting '=value' for error-format" << endl; + else if ( strcmp( eq, "gnu" ) == 0 ) + errorFormat = ErrorFormatGNU; + else if ( strcmp( eq, "msvc" ) == 0 ) + errorFormat = ErrorFormatMSVC; + else + error() << "invalid value for error-format" << endl; + } + else if ( strcmp( arg, "rbx" ) == 0 ) + rubyImpl = Rubinius; + else { + error() << "--" << pc.paramArg << + " is an invalid argument" << endl; + } + free( arg ); + break; + } + + /* Passthrough args. */ + case 'T': + if ( pc.paramArg[0] == '0' ) + codeStyle = GenTables; + else if ( pc.paramArg[0] == '1' ) + codeStyle = GenFTables; + else { + error() << "-T" << pc.paramArg[0] << + " is an invalid argument" << endl; + exit(1); + } + break; + case 'F': + if ( pc.paramArg[0] == '0' ) + codeStyle = GenFlat; + else if ( pc.paramArg[0] == '1' ) + codeStyle = GenFFlat; + else { + error() << "-F" << pc.paramArg[0] << + " is an invalid argument" << endl; + exit(1); + } + break; + case 'G': + if ( pc.paramArg[0] == '0' ) + codeStyle = GenGoto; + else if ( pc.paramArg[0] == '1' ) + codeStyle = GenFGoto; + else if ( pc.paramArg[0] == '2' ) + codeStyle = GenIpGoto; + else { + error() << "-G" << pc.paramArg[0] << + " is an invalid argument" << endl; + exit(1); + } + break; + case 'P': + codeStyle = GenSplit; + numSplitPartitions = atoi( pc.paramArg ); + break; + + case 'p': + displayPrintables = true; + break; + + case 'L': + noLineDirectives = true; + break; + } + break; + + case ParamCheck::invalid: + error() << "-" << pc.parameter << " is an invalid argument" << endl; + break; + + case ParamCheck::noparam: + /* It is interpreted as an input file. */ + if ( *pc.curArg == 0 ) + error() << "a zero length input file name was given" << endl; + else if ( id.inputFileName != 0 ) + error() << "more than one input file name was given" << endl; + else { + /* OK, Remember the filename. */ + id.inputFileName = pc.curArg; + } + break; + } + } +} + +void process( InputData &id ) +{ + /* Open the input file for reading. */ + assert( id.inputFileName != 0 ); + ifstream *inFile = new ifstream( id.inputFileName ); + if ( ! inFile->is_open() ) + error() << "could not open " << id.inputFileName << " for reading" << endp; + + /* Used for just a few things. */ + std::ostringstream hostData; + + /* Make the first input item. */ + InputItem *firstInputItem = new InputItem; + firstInputItem->type = InputItem::HostData; + firstInputItem->loc.fileName = id.inputFileName; + firstInputItem->loc.line = 1; + firstInputItem->loc.col = 1; + id.inputItems.append( firstInputItem ); + + Scanner scanner( id, id.inputFileName, *inFile, 0, 0, 0, false ); + scanner.do_scan(); + + /* Finished, final check for errors.. */ + if ( gblErrorCount > 0 ) + exit(1); + + /* Now send EOF to all parsers. */ + id.terminateAllParsers(); + + /* Bail on above error. */ + if ( gblErrorCount > 0 ) + exit(1); + + /* Locate the backend program */ + /* Compiles machines. */ + id.prepareMachineGen(); + + if ( gblErrorCount > 0 ) + exit(1); + + id.makeOutputStream(); + + /* Generates the reduced machine, which we use to write output. */ + if ( !generateXML ) { + id.generateReduced(); + + if ( gblErrorCount > 0 ) + exit(1); + } + + id.verifyWritesHaveData(); + if ( gblErrorCount > 0 ) + exit(1); + + /* + * From this point on we should not be reporting any errors. + */ + + id.openOutput(); + id.writeOutput(); + + /* Close the input and the intermediate file. */ + delete inFile; + + /* If writing to a file, delete the ostream, causing it to flush. + * Standard out is flushed automatically. */ + if ( id.outputFileName != 0 ) { + delete id.outStream; + delete id.outFilter; + } + + assert( gblErrorCount == 0 ); +} + +char *makeIntermedTemplate( const char *baseFileName ) +{ + char *result = 0; + const char *templ = "ragel-XXXXXX.xml"; + const char *lastSlash = strrchr( baseFileName, '/' ); + if ( lastSlash == 0 ) { + result = new char[strlen(templ)+1]; + strcpy( result, templ ); + } + else { + int baseLen = lastSlash - baseFileName + 1; + result = new char[baseLen + strlen(templ) + 1]; + memcpy( result, baseFileName, baseLen ); + strcpy( result+baseLen, templ ); + } + return result; +}; + +/* Main, process args and call yyparse to start scanning input. */ +int main( int argc, const char **argv ) +{ + InputData id; + + processArgs( argc, argv, id ); + + /* Require an input file. If we use standard in then we won't have a file + * name on which to base the output. */ + if ( id.inputFileName == 0 ) + error() << "no input file given" << endl; + + /* Bail on argument processing errors. */ + if ( gblErrorCount > 0 ) + exit(1); + + /* Make sure we are not writing to the same file as the input file. */ + if ( id.inputFileName != 0 && id.outputFileName != 0 && + strcmp( id.inputFileName, id.outputFileName ) == 0 ) + { + error() << "output file \"" << id.outputFileName << + "\" is the same as the input file" << endp; + } + + for (char* p = (char*)id.inputFileName; *p != 0; p++) { + if (*p == '\\') + *p = '/'; + } + + process( id ); + + return 0; +} diff --git a/contrib/tools/ragel6/mlcodegen.cpp b/contrib/tools/ragel6/mlcodegen.cpp new file mode 100644 index 0000000000..166e45347c --- /dev/null +++ b/contrib/tools/ragel6/mlcodegen.cpp @@ -0,0 +1,744 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "mlcodegen.h" +#include "redfsm.h" +#include "gendata.h" +#include <sstream> +#include <iomanip> +#include <string> +#include <assert.h> + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; + +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + +void ocamlLineDirective( ostream &out, const char *fileName, int line ) +{ + if ( noLineDirectives ) + return; + + /* Write the line info for to the input file. */ + out << "# " << line << " \""; + for ( const char *pc = fileName; *pc != 0; pc++ ) { + if ( *pc == '\\' || *pc == '"' ) + out << "\\"; + out << *pc; + } + out << "\"\n"; +} + +void OCamlCodeGen::genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + ocamlLineDirective( out, filter->fileName, filter->line + 1 ); +} + + +/* Init code gen with in parameters. */ +OCamlCodeGen::OCamlCodeGen( ostream &out ) +: + CodeGenData(out) +{ +} + +unsigned int OCamlCodeGen::arrayTypeSize( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + return arrayType->size; +} + +string OCamlCodeGen::ARRAY_TYPE( unsigned long maxVal ) +{ + return ARRAY_TYPE( maxVal, false ); +} + +string OCamlCodeGen::ARRAY_TYPE( unsigned long maxVal, bool forceSigned ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType; + if (forceSigned) + arrayType = keyOps->typeSubsumes(true, maxValLL); + else + arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + + string ret = arrayType->data1; + if ( arrayType->data2 != 0 ) { + ret += " "; + ret += arrayType->data2; + } + return ret; +} + +/* Write out the fsm name. */ +string OCamlCodeGen::FSM_NAME() +{ + return fsmName; +} + +/* Emit the offset of the start state as a decimal integer. */ +string OCamlCodeGen::START_STATE_ID() +{ + ostringstream ret; + ret << redFsm->startState->id; + return ret.str(); +}; + +/* Write out the array of actions. */ +std::ostream &OCamlCodeGen::ACTIONS_ARRAY() +{ + out << "\t0; "; + int totalActions = 1; + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + /* Write out the length, which will never be the last character. */ + out << act->key.length() << ARR_SEP(); + /* Put in a line break every 8 */ + if ( totalActions++ % 8 == 7 ) + out << "\n\t"; + + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) { + out << item->value->actionId; + if ( ! (act.last() && item.last()) ) + out << ARR_SEP(); + + /* Put in a line break every 8 */ + if ( totalActions++ % 8 == 7 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +/* +string OCamlCodeGen::ACCESS() +{ + ostringstream ret; + if ( accessExpr != 0 ) + INLINE_LIST( ret, accessExpr, 0, false ); + return ret.str(); +} +*/ + +string OCamlCodeGen::make_access(char const* name, GenInlineList* x, bool prefix = true) +{ + ostringstream ret; + if ( x == 0 ) + { + if (prefix && accessExpr != 0) + { + INLINE_LIST( ret, accessExpr, 0, false); + ret << name; + } + else + ret << name << ".contents"; // ref cell + } + else { + ret << "("; + INLINE_LIST( ret, x, 0, false ); + ret << ")"; + } + return ret.str(); +} + +string OCamlCodeGen::P() { return make_access("p", pExpr, false); } +string OCamlCodeGen::PE() { return make_access("pe", peExpr, false); } +string OCamlCodeGen::vEOF() { return make_access("eof", eofExpr, false); } +string OCamlCodeGen::vCS() { return make_access("cs", csExpr); } +string OCamlCodeGen::TOP() { return make_access("top", topExpr); } +string OCamlCodeGen::STACK() { return make_access("stack", stackExpr); } +string OCamlCodeGen::ACT() { return make_access("act", actExpr); } +string OCamlCodeGen::TOKSTART() { return make_access("ts", tokstartExpr); } +string OCamlCodeGen::TOKEND() { return make_access("te", tokendExpr); } + +string OCamlCodeGen::GET_WIDE_KEY() +{ + if ( redFsm->anyConditions() ) + return "_widec"; + else + { ostringstream ret; ret << "Char.code " << GET_KEY(); return ret.str(); } +} + +string OCamlCodeGen::GET_WIDE_KEY( RedStateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) + return "_widec"; + else + { ostringstream ret; ret << "Char.code " << GET_KEY(); return ret.str(); } +} + +/* Write out level number of tabs. Makes the nested binary search nice + * looking. */ +string OCamlCodeGen::TABS( int level ) +{ + string result; + while ( level-- > 0 ) + result += "\t"; + return result; +} + +/* Write out a key from the fsm code gen. Depends on wether or not the key is + * signed. */ +string OCamlCodeGen::KEY( Key key ) +{ + ostringstream ret; + if ( keyOps->isSigned || !hostLang->explicitUnsigned ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal() << 'u'; + return ret.str(); +} + +string OCamlCodeGen::ALPHA_KEY( Key key ) +{ + ostringstream ret; + ret << key.getVal(); + /* + if (key.getVal() > 0xFFFF) { + ret << key.getVal(); + } else { + ret << "'\\u" << std::hex << std::setw(4) << std::setfill('0') << + key.getVal() << "'"; + } + */ + //ret << "(char) " << key.getVal(); + return ret.str(); +} + +void OCamlCodeGen::EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ) +{ +// The parser gives fexec two children. + ret << "begin " << P() << " <- "; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << " - 1 end; "; +} + +void OCamlCodeGen::LM_SWITCH( ostream &ret, GenInlineItem *item, + int targState, int inFinish ) +{ + bool catch_all = false; + ret << + " begin match " << ACT() << " with\n"; + + for ( GenInlineList::Iter lma = *item->children; lma.lte(); lma++ ) { + /* Write the case label, the action and the case break. */ + if ( lma->lmId < 0 ) + { + catch_all = true; + ret << " | _ ->\n"; + } + else + ret << " | " << lma->lmId << " ->\n"; + + /* Write the block and close it off. */ + ret << " begin "; + INLINE_LIST( ret, lma->children, targState, inFinish ); + ret << " end\n"; + } + + if (!catch_all) + ret << " | _ -> assert false\n"; + + ret << + " end;\n" + "\t"; +} + +void OCamlCodeGen::SET_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " <- " << item->lmId << "; "; +} + +void OCamlCodeGen::SET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + /* The tokend action sets tokend. */ + ret << TOKEND() << " <- " << P(); + if ( item->offset != 0 ) + out << "+" << item->offset; + out << "; "; +} + +void OCamlCodeGen::GET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + ret << TOKEND(); +} + +void OCamlCodeGen::INIT_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " <- " << NULL_ITEM() << "; "; +} + +void OCamlCodeGen::INIT_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " <- 0;"; +} + +void OCamlCodeGen::SET_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " <- " << P() << "; "; +} + +void OCamlCodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << "begin "; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << " end"; + } +} + + +/* Write out an inline tree structure. Walks the list and possibly calls out + * to virtual functions than handle language specific items in the tree. */ +void OCamlCodeGen::INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Text: + ret << item->data; + break; + case GenInlineItem::Goto: + GOTO( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Call: + CALL( ret, item->targState->id, targState, inFinish ); + break; + case GenInlineItem::Next: + NEXT( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Ret: + RET( ret, inFinish ); + break; + case GenInlineItem::PChar: + ret << P(); + break; + case GenInlineItem::Char: + ret << GET_KEY(); + break; + case GenInlineItem::Hold: + ret << P() << " <- " << P() << " - 1; "; + break; + case GenInlineItem::Exec: + EXEC( ret, item, targState, inFinish ); + break; + case GenInlineItem::Curs: + CURS( ret, inFinish ); + break; + case GenInlineItem::Targs: + TARGS( ret, inFinish, targState ); + break; + case GenInlineItem::Entry: + ret << item->targState->id; + break; + case GenInlineItem::GotoExpr: + GOTO_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::CallExpr: + CALL_EXPR( ret, item, targState, inFinish ); + break; + case GenInlineItem::NextExpr: + NEXT_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::LmSwitch: + LM_SWITCH( ret, item, targState, inFinish ); + break; + case GenInlineItem::LmSetActId: + SET_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokEnd: + SET_TOKEND( ret, item ); + break; + case GenInlineItem::LmGetTokEnd: + GET_TOKEND( ret, item ); + break; + case GenInlineItem::LmInitTokStart: + INIT_TOKSTART( ret, item ); + break; + case GenInlineItem::LmInitAct: + INIT_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokStart: + SET_TOKSTART( ret, item ); + break; + case GenInlineItem::SubAction: + SUB_ACTION( ret, item, targState, inFinish ); + break; + case GenInlineItem::Break: + BREAK( ret, targState ); + break; + } + } +} +/* Write out paths in line directives. Escapes any special characters. */ +string OCamlCodeGen::LDIR_PATH( char *path ) +{ + ostringstream ret; + for ( char *pc = path; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + ret << "\\\\"; + else + ret << *pc; + } + return ret.str(); +} + +void OCamlCodeGen::ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ) +{ + /* Write the preprocessor line info for going into the source file. */ + ocamlLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + ret << "\t\tbegin "; + INLINE_LIST( ret, action->inlineList, targState, inFinish ); + ret << " end;\n"; +} + +void OCamlCodeGen::CONDITION( ostream &ret, GenAction *condition ) +{ + ret << "\n"; + ocamlLineDirective( ret, condition->loc.fileName, condition->loc.line ); + INLINE_LIST( ret, condition->inlineList, 0, false ); +} + +string OCamlCodeGen::ERROR_STATE() +{ + ostringstream ret; + if ( redFsm->errState != 0 ) + ret << redFsm->errState->id; + else + ret << "-1"; + return ret.str(); +} + +string OCamlCodeGen::FIRST_FINAL_STATE() +{ + ostringstream ret; + if ( redFsm->firstFinState != 0 ) + ret << redFsm->firstFinState->id; + else + ret << redFsm->nextStateId; + return ret.str(); +} + +void OCamlCodeGen::writeInit() +{ + out << " begin\n"; + + if ( !noCS ) + out << "\t" << vCS() << " <- " << START() << ";\n"; + + /* If there are any calls, then the stack top needs initialization. */ + if ( redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\t" << TOP() << " <- 0;\n"; + + if ( hasLongestMatch ) { + out << + " " << TOKSTART() << " <- " << NULL_ITEM() << ";\n" + " " << TOKEND() << " <- " << NULL_ITEM() << ";\n" + " " << ACT() << " <- 0;\n"; + } + out << " end;\n"; +} + +string OCamlCodeGen::PRE_INCR(string val) +{ + ostringstream ret; + ret << "(" << val << " <- " << val << " + 1; " << val << ")"; + return ret.str(); +} + +string OCamlCodeGen::POST_INCR(string val) +{ + ostringstream ret; + ret << "(let temp = " << val << " in " << val << " <- " << val << " + 1; temp)"; + return ret.str(); +} + +string OCamlCodeGen::PRE_DECR(string val) +{ + ostringstream ret; + ret << "(" << val << " <- " << val << " - 1; " << val << ")"; + return ret.str(); +} + +string OCamlCodeGen::POST_DECR(string val) +{ + ostringstream ret; + ret << "(let temp = " << val << " in " << val << " <- " << val << " - 1; temp)"; + return ret.str(); +} + +string OCamlCodeGen::DATA_PREFIX() +{ + if ( data_prefix.empty() ) // init + { + data_prefix = string(fsmName) + "_"; + if (data_prefix.size() > 0) + data_prefix[0] = ::tolower(data_prefix[0]); // uncapitalize + } + if ( !noPrefix ) + return data_prefix; + return ""; +} + +/* Emit the alphabet data type. */ +string OCamlCodeGen::ALPH_TYPE() +{ + string ret = keyOps->alphType->data1; + if ( keyOps->alphType->data2 != 0 ) { + ret += " "; + ret += + keyOps->alphType->data2; + } + return ret; +} + +/* Emit the alphabet data type. */ +string OCamlCodeGen::WIDE_ALPH_TYPE() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + ret = ALPH_TYPE(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + assert( wideType != 0 ); + + ret = wideType->data1; + if ( wideType->data2 != 0 ) { + ret += " "; + ret += wideType->data2; + } + } + return ret; +} + +void OCamlCodeGen::STATE_IDS() +{ + if ( redFsm->startState != 0 ) + STATIC_VAR( "int", START() ) << " = " << START_STATE_ID() << TOP_SEP (); + + if ( !noFinal ) + STATIC_VAR( "int" , FIRST_FINAL() ) << " = " << FIRST_FINAL_STATE() << TOP_SEP(); + + if ( !noError ) + STATIC_VAR( "int", ERROR() ) << " = " << ERROR_STATE() << TOP_SEP(); + + out << "\n"; + + if ( !noEntry && entryPointNames.length() > 0 ) { + for ( EntryNameVect::Iter en = entryPointNames; en.lte(); en++ ) { + STATIC_VAR( "int", DATA_PREFIX() + "en_" + *en ) << + " = " << entryPointIds[en.pos()] << TOP_SEP(); + } + out << "\n"; + } +} + + +void OCamlCodeGen::writeStart() +{ + out << START_STATE_ID(); +} + +void OCamlCodeGen::writeFirstFinal() +{ + out << FIRST_FINAL_STATE(); +} + +void OCamlCodeGen::writeError() +{ + out << ERROR_STATE(); +} + +string OCamlCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use simple dereference. */ + ret << "data.[" << P() << "]"; + } + return ret.str(); +} +string OCamlCodeGen::NULL_ITEM() +{ + return "-1"; +} + +string OCamlCodeGen::POINTER() +{ + // XXX C# has no pointers + // multiple items seperated by commas can also be pointer types. + return " "; +} + +string OCamlCodeGen::PTR_CONST() +{ + return ""; +} + +std::ostream &OCamlCodeGen::OPEN_ARRAY( string type, string name ) +{ + out << "let " << name << " : " << type << " array = [|" << endl; + return out; +} + +std::ostream &OCamlCodeGen::CLOSE_ARRAY() +{ + return out << "|]" << TOP_SEP(); +} + +string OCamlCodeGen::TOP_SEP() +{ + return "\n"; // original syntax +} + +string OCamlCodeGen::ARR_SEP() +{ + return "; "; +} + +string OCamlCodeGen::AT(const string& array, const string& index) +{ + ostringstream ret; + ret << array << ".(" << index << ")"; + return ret.str(); +} + +std::ostream &OCamlCodeGen::STATIC_VAR( string type, string name ) +{ + out << "let " << name << " : " << type; + return out; +} + +string OCamlCodeGen::ARR_OFF( string ptr, string offset ) +{ + // XXX C# can't do pointer arithmetic + return "&" + ptr + "[" + offset + "]"; +} + +string OCamlCodeGen::CAST( string type ) +{ + return ""; +// return "(" + type + ")"; +} + +string OCamlCodeGen::UINT( ) +{ + return "uint"; +} + +std::ostream &OCamlCodeGen::SWITCH_DEFAULT() +{ + out << " | _ -> ()\n"; + return out; +} + +string OCamlCodeGen::CTRL_FLOW() +{ + return "if true then "; +} + +void OCamlCodeGen::finishRagelDef() +{ + if ( codeStyle == GenGoto || codeStyle == GenFGoto || + codeStyle == GenIpGoto || codeStyle == GenSplit ) + { + /* For directly executable machines there is no required state + * ordering. Choose a depth-first ordering to increase the + * potential for fall-throughs. */ + redFsm->depthFirstOrdering(); + } + else { + /* The frontend will do this for us, but it may be a good idea to + * force it if the intermediate file is edited. */ + redFsm->sortByStateId(); + } + + /* Choose default transitions and the single transition. */ + redFsm->chooseDefaultSpan(); + + /* Maybe do flat expand, otherwise choose single. */ + if ( codeStyle == GenFlat || codeStyle == GenFFlat ) + redFsm->makeFlat(); + else + redFsm->chooseSingle(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + if ( codeStyle == GenSplit ) + redFsm->partitionFsm( numSplitPartitions ); + + if ( codeStyle == GenIpGoto || codeStyle == GenSplit ) + redFsm->setInTrans(); + + /* Anlayze Machine will find the final action reference counts, among + * other things. We will use these in reporting the usage + * of fsm directives in action code. */ + analyzeMachine(); + + /* Determine if we should use indicies. */ + calcIndexSize(); +} + +ostream &OCamlCodeGen::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &OCamlCodeGen::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + diff --git a/contrib/tools/ragel6/mlcodegen.h b/contrib/tools/ragel6/mlcodegen.h new file mode 100644 index 0000000000..9ca49164de --- /dev/null +++ b/contrib/tools/ragel6/mlcodegen.h @@ -0,0 +1,206 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _MLCODEGEN_H +#define _MLCODEGEN_H + +#include <iostream> +#include <string> +#include <stdio.h> +#include "common.h" +#include "gendata.h" + +using std::string; +using std::ostream; + +/* Forwards. */ +/* +struct RedFsmAp; +struct RedStateAp; +struct CodeGenData; +struct GenAction; +struct NameInst; +struct GenInlineItem; +struct GenInlineList; +struct RedAction; +struct LongestMatch; +struct LongestMatchPart; +*/ + +/* Integer array line length. */ +#define IALL 8 + +//string itoa( int i ); + +/* + * class OCamlCodeGen + */ +class OCamlCodeGen : public CodeGenData +{ +public: + OCamlCodeGen( ostream &out ); + virtual ~OCamlCodeGen() {} + + virtual void finishRagelDef(); + virtual void writeInit(); + virtual void writeStart(); + virtual void writeFirstFinal(); + virtual void writeError(); + +protected: + string data_prefix; + + string FSM_NAME(); + string START_STATE_ID(); + ostream &ACTIONS_ARRAY(); + string GET_WIDE_KEY(); + string GET_WIDE_KEY( RedStateAp *state ); + string TABS( int level ); + string KEY( Key key ); + string ALPHA_KEY( Key key ); + string LDIR_PATH( char *path ); + void ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ); + void CONDITION( ostream &ret, GenAction *condition ); + string ALPH_TYPE(); + string WIDE_ALPH_TYPE(); + string ARRAY_TYPE( unsigned long maxVal ); + string ARRAY_TYPE( unsigned long maxVal, bool forceSigned ); + + virtual string ARR_OFF( string ptr, string offset ); + virtual string CAST( string type ); + virtual string UINT(); + virtual string NULL_ITEM(); + virtual string POINTER(); + virtual string GET_KEY(); + virtual ostream &SWITCH_DEFAULT(); + + string P(); + string PE(); + string vEOF(); + +// string ACCESS(); + string vCS(); + string STACK(); + string TOP(); + string TOKSTART(); + string TOKEND(); + string ACT(); + + // ++x + string PRE_INCR(string); + string PRE_DECR(string); + + // x++ + string POST_INCR(string); + string POST_DECR(string); + + string DATA_PREFIX(); + string PM() { return "_" + DATA_PREFIX() + "partition_map"; } + string C() { return "_" + DATA_PREFIX() + "cond_spaces"; } + string CK() { return "_" + DATA_PREFIX() + "cond_keys"; } + string K() { return "_" + DATA_PREFIX() + "trans_keys"; } + string I() { return "_" + DATA_PREFIX() + "indicies"; } + string CO() { return "_" + DATA_PREFIX() + "cond_offsets"; } + string KO() { return "_" + DATA_PREFIX() + "key_offsets"; } + string IO() { return "_" + DATA_PREFIX() + "index_offsets"; } + string CL() { return "_" + DATA_PREFIX() + "cond_lengths"; } + string SL() { return "_" + DATA_PREFIX() + "single_lengths"; } + string RL() { return "_" + DATA_PREFIX() + "range_lengths"; } + string A() { return "_" + DATA_PREFIX() + "actions"; } + string TA() { return "_" + DATA_PREFIX() + "trans_actions"; } + string TT() { return "_" + DATA_PREFIX() + "trans_targs"; } + string TSA() { return "_" + DATA_PREFIX() + "to_state_actions"; } + string FSA() { return "_" + DATA_PREFIX() + "from_state_actions"; } + string EA() { return "_" + DATA_PREFIX() + "eof_actions"; } + string ET() { return "_" + DATA_PREFIX() + "eof_trans"; } + string SP() { return "_" + DATA_PREFIX() + "key_spans"; } + string CSP() { return "_" + DATA_PREFIX() + "cond_key_spans"; } + string TYPE_STATE() { return "_" + DATA_PREFIX() + "state"; } + string START() { return DATA_PREFIX() + "start"; } + string ERROR() { return DATA_PREFIX() + "error"; } + string FIRST_FINAL() { return DATA_PREFIX() + "first_final"; } + string CTXDATA() { return DATA_PREFIX() + "ctxdata"; } + + void INLINE_LIST( ostream &ret, GenInlineList *inlineList, int targState, bool inFinish ); + virtual void GOTO( ostream &ret, int gotoDest, bool inFinish ) = 0; + virtual void CALL( ostream &ret, int callDest, int targState, bool inFinish ) = 0; + virtual void NEXT( ostream &ret, int nextDest, bool inFinish ) = 0; + virtual void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, + int targState, bool inFinish ) = 0; + virtual void RET( ostream &ret, bool inFinish ) = 0; + virtual void BREAK( ostream &ret, int targState ) = 0; + virtual void CURS( ostream &ret, bool inFinish ) = 0; + virtual void TARGS( ostream &ret, bool inFinish, int targState ) = 0; + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ); + void STATE_IDS(); + + string ERROR_STATE(); + string FIRST_FINAL_STATE(); + + virtual string PTR_CONST(); + virtual ostream &OPEN_ARRAY( string type, string name ); + virtual ostream &CLOSE_ARRAY(); + virtual ostream &STATIC_VAR( string type, string name ); + + virtual string CTRL_FLOW(); + + // toplevel phrase separator + string TOP_SEP(); + // array elements separator + string ARR_SEP(); + // access array + string AT(const string& array, const string& index); + + string make_access(char const* name, GenInlineList* x, bool prefix); + + ostream &source_warning(const InputLoc &loc); + ostream &source_error(const InputLoc &loc); + + unsigned int arrayTypeSize( unsigned long maxVal ); + + bool outLabelUsed; + bool testEofUsed; + bool againLabelUsed; + bool useIndicies; + +public: + /* Determine if we should use indicies. */ + virtual void calcIndexSize() {} + + void genLineDirective( ostream &out ); +}; + +#define MAX(a, b) (a > b ? a : b) + +#endif diff --git a/contrib/tools/ragel6/mlfflat.cpp b/contrib/tools/ragel6/mlfflat.cpp new file mode 100644 index 0000000000..2e14543a37 --- /dev/null +++ b/contrib/tools/ragel6/mlfflat.cpp @@ -0,0 +1,419 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "mlfflat.h" +#include "redfsm.h" +#include "gendata.h" + +std::ostream &OCamlFFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &OCamlFFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &OCamlFFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + +/* Write out the function for a transition. */ +std::ostream &OCamlFFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &OCamlFFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\t| " << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &OCamlFFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\t| " << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlFFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\t| " << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &OCamlFFlatCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\t| " << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +void OCamlFFlatCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); + + out << "type " << TYPE_STATE() << " = { mutable keys : int; mutable trans : int; }" + << TOP_SEP(); + + out << "exception Goto_match" << TOP_SEP(); + out << "exception Goto_again" << TOP_SEP(); + out << "exception Goto_eof_trans" << TOP_SEP(); +} + +void OCamlFFlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " begin\n"; +// " " << slenType << " _slen"; + +// if ( redFsm->anyRegCurStateRef() ) +// out << ", _ps"; + +// out << ";\n"; +// out << " " << transType << " _trans"; + +// if ( redFsm->anyConditions() ) +// out << ", _cond"; + +// out << ";\n"; + +// out << +// " " << "int _keys;\n" +// " " << indsType << " _inds;\n"; + /* + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxIndex) << POINTER() << "_inds;\n";*/ + + out << + " let state = { keys = 0; trans = 0; } in\n" + " let rec do_start () =\n"; + +// if ( redFsm->anyConditions() ) { +// out << +// " " << condsType << " _conds;\n" +// " " << WIDE_ALPH_TYPE() << " _widec;\n"; +// } + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " = " << PE() << " then\n" + " do_test_eof ()\n" + "\telse\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " = " << redFsm->errState->id << " then\n" + " do_out ()\n" + "\telse\n"; + } + out << "\tdo_resume ()\n"; + + out << "and do_resume () =\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " begin match " << AT( FSA(), vCS() ) << " with\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + out << "\tbegin try\n"; + LOCATE_TRANS(); + out << "\twith Goto_match -> () end;\n"; + + out << "\tdo_eof_trans ()\n"; + +// if ( redFsm->anyEofTrans() ) + out << "and do_eof_trans () =\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " let ps = " << vCS() << " in\n"; + + out << + " " << vCS() << " <- " << AT( TT() ,"state.trans" ) << ";\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " begin try if " << AT( TA() , "state.trans" ) << " = 0 then\n" + " raise Goto_again;\n" + "\n" + " match " << AT( TA(), "state.trans" ) << " with\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " with Goto_again -> () end;\n" + "\n"; + } + out << "\tdo_again ()\n"; + +// if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || +// redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\tand do_again () =\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " begin match " << AT( TSA(), vCS() ) << " with\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " match " << vCS() << " with\n" + "\t| " << redFsm->errState->id << " -> do_out ()\n" + "\t| _ ->\n"; + } + + out << "\t" << P() << " <- " << P() << " + 1;\n"; + + if ( !noEnd ) { + out << + " if " << P() << " <> " << PE() << " then\n" + " do_resume ()\n" + "\telse do_test_eof ()\n"; + } + else { + out << + " do_resume ()\n"; + } + +// if ( testEofUsed ) + out << "and do_test_eof () =\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " = " << vEOF() << " then\n" + " begin try\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << AT( ET(), vCS() ) << " > 0 then\n" + " begin\n" + " state.trans <- " << CAST(transType) << "(" << AT( ET(), vCS() ) << " - 1);\n" + " raise Goto_eof_trans;\n" + " end;\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " begin match " << AT( EA(), vCS() ) << " with\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end\n"; + } + + out << + " with Goto_again -> do_again ()\n" + " | Goto_eof_trans -> do_eof_trans () end\n" + "\n"; + } + else + { + out << "\t()\n"; + } + + if ( outLabelUsed ) + out << " and do_out () = ()\n"; + + out << "\tin do_start ()\n"; + out << " end;\n"; +} + diff --git a/contrib/tools/ragel6/mlfflat.h b/contrib/tools/ragel6/mlfflat.h new file mode 100644 index 0000000000..242e6b911a --- /dev/null +++ b/contrib/tools/ragel6/mlfflat.h @@ -0,0 +1,55 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _MLFFLAT_H +#define _MLFFLAT_H + +#include <iostream> +#include "mlflat.h" + +/* Forwards. */ +//struct CodeGenData; + +/* + * OCamlFFlatCodeGen + */ +class OCamlFFlatCodeGen : public OCamlFlatCodeGen +{ +public: + OCamlFFlatCodeGen( ostream &out ) : OCamlFlatCodeGen(out) {} +private: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/contrib/tools/ragel6/mlfgoto.cpp b/contrib/tools/ragel6/mlfgoto.cpp new file mode 100644 index 0000000000..b66d66ec2c --- /dev/null +++ b/contrib/tools/ragel6/mlfgoto.cpp @@ -0,0 +1,311 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "mlfgoto.h" +#include "redfsm.h" +#include "gendata.h" +#include "bstmap.h" + +std::ostream &OCamlFGotoCodeGen::EXEC_ACTIONS() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* We are at the start of a glob, write the case. */ + out << "and f" << redAct->actListId << " () =\n"; + out << "\tbegin try\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\twith Goto_again -> () end;\n"; + out << "\tdo_again ()\n"; + } + } + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &OCamlFGotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\t|" << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + +// out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &OCamlFGotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\t| " << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + +// out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlFGotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\t| " << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + +// out << "\tbreak;\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &OCamlFGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\t| " << st->id << " -> "; + + /* Jump to the func. */ + out << "f" << st->eofAction->actListId << " ()\n"; + } + } + + return out; +} + +unsigned int OCamlFGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + return act; +} + +unsigned int OCamlFGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + return act; +} + +unsigned int OCamlFGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + return act; +} + +void OCamlFGotoCodeGen::writeData() +{ + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); + + out << "exception Goto_again" << TOP_SEP(); +} + +void OCamlFGotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " begin\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " let _ps = ref 0 in\n"; + + if ( redFsm->anyConditions() ) + out << " let _widec : " << WIDE_ALPH_TYPE() << " = ref 0 in\n"; + + out << "\n"; + out << "\tlet rec do_start () =\n"; + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " = " << PE() << " then\n" + " do_test_eof ()\n" + "\telse\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " = " << redFsm->errState->id << " then\n" + " do_out ()\n" + "\telse\n"; + } + out << "\tdo_resume ()\n"; + + out << "and do_resume () =\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " begin match " << AT(FSA(),vCS()) << " with\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + "\n"; + } + + out << + " begin match " << vCS() << " with\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " end\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_ACTIONS() << "\n"; + + out << "\tand do_again () =\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " begin match " << AT(TSA(), vCS()) << " with\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " match " << vCS() << " with\n" << + " | " << redFsm->errState->id << " -> do_out ()\n" + " | _ ->\n"; + } + + out << "\t" << P() << " <- " << P() << " + 1;\n"; + + if ( !noEnd ) { + out << + " if " << P() << " <> " << PE() << " then\n" + " do_resume ()\n" + "\telse do_test_eof ()\n"; + } + else { + out << + " do_resume ()\n"; + } + +// if ( testEofUsed ) + out << "and do_test_eof () =\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " = " << vEOF() << " then\n" + " begin\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " match " << vCS() << " with\n"; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " | " << st->id << " -> tr" << st->eofTrans->id << " ()\n"; + } + + SWITCH_DEFAULT() << ";\n"; // fall through + } + + if ( redFsm->anyEofActions() ) { + out << + " try match " << AT(EA(), vCS()) << " with\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " \n" + " with Goto_again -> do_again () \n"; + } + + out << + " end\n" + "\n"; + } + else + out << "\t()\n"; + + if ( outLabelUsed ) + out << "\tand do_out () = ()\n"; + + out << "\tin do_start ()\n"; + out << " end;\n"; +} diff --git a/contrib/tools/ragel6/mlfgoto.h b/contrib/tools/ragel6/mlfgoto.h new file mode 100644 index 0000000000..bccbd5523f --- /dev/null +++ b/contrib/tools/ragel6/mlfgoto.h @@ -0,0 +1,51 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _MLFGOTO_H +#define _MLFGOTO_H + +#include <iostream> +#include "mlgoto.h" + +/* + * class OCamlFGotoCodeGen + */ +class OCamlFGotoCodeGen : virtual public OCamlGotoCodeGen +{ +public: + OCamlFGotoCodeGen( ostream &out ) : OCamlCodeGen(out), OCamlGotoCodeGen(out) {} + + std::ostream &EXEC_ACTIONS(); + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &FINISH_CASES(); + std::ostream &EOF_ACTION_SWITCH(); + unsigned int TO_STATE_ACTION( RedStateAp *state ); + unsigned int FROM_STATE_ACTION( RedStateAp *state ); + unsigned int EOF_ACTION( RedStateAp *state ); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/contrib/tools/ragel6/mlflat.cpp b/contrib/tools/ragel6/mlflat.cpp new file mode 100644 index 0000000000..b97aa15a8c --- /dev/null +++ b/contrib/tools/ragel6/mlflat.cpp @@ -0,0 +1,911 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <sstream> +#include "ragel.h" +#include "mlflat.h" +#include "redfsm.h" +#include "gendata.h" + +std::ostream &OCamlFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &OCamlFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &OCamlFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + +std::ostream &OCamlFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &OCamlFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, false ); + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, false ); + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, true ); + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &OCamlFlatCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, false ); + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &OCamlFlatCodeGen::FLAT_INDEX_OFFSET() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + if ( st->transList != 0 ) + curIndOffset += keyOps->span( st->lowKey, st->highKey ); + + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << "\n"; + return out; +} + +std::ostream &OCamlFlatCodeGen::KEY_SPANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->transList != 0 ) + span = keyOps->span( st->lowKey, st->highKey ); + out << span; + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &OCamlFlatCodeGen::TO_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &OCamlFlatCodeGen::FROM_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &OCamlFlatCodeGen::EOF_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &OCamlFlatCodeGen::EOF_TRANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &OCamlFlatCodeGen::COND_KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just cond low key and cond high key. */ + out << ALPHA_KEY( st->condLowKey ) << ARR_SEP(); + out << ALPHA_KEY( st->condHighKey ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << /*"(char) " <<*/ 0 << "\n"; + return out; +} + +std::ostream &OCamlFlatCodeGen::COND_KEY_SPANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->condList != 0 ) + span = keyOps->span( st->condLowKey, st->condHighKey ); + out << span; + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &OCamlFlatCodeGen::CONDS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->condList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + if ( st->condList[pos] != 0 ) + out << st->condList[pos]->condSpaceId + 1 << ARR_SEP(); + else + out << "0" << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &OCamlFlatCodeGen::COND_INDEX_OFFSET() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + if ( st->condList != 0 ) + curIndOffset += keyOps->span( st->condLowKey, st->condHighKey ); + } + out << "\n"; + return out; +} + + +std::ostream &OCamlFlatCodeGen::KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just low key and high key. */ + out << ALPHA_KEY( st->lowKey ) << ARR_SEP(); + out << ALPHA_KEY( st->highKey ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << /*"(char) " <<*/ 0 << "\n"; + return out; +} + +std::ostream &OCamlFlatCodeGen::INDICIES() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->transList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + out << st->transList[pos]->id << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) + out << st->defTrans->id << ARR_SEP(); + + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &OCamlFlatCodeGen::TRANS_TARGS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Record the position, need this for eofTrans. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id; + if ( t < redFsm->transSet.length()-1 ) { + out << ARR_SEP(); + if ( ++totalStates % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + + +std::ostream &OCamlFlatCodeGen::TRANS_ACTIONS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + if ( t < redFsm->transSet.length()-1 ) { + out << ARR_SEP(); + if ( ++totalAct % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + +void OCamlFlatCodeGen::LOCATE_TRANS() +{ + std::ostringstream temp; + temp << "inds + (\n" + " if slen > 0 && " << AT( K(), "keys" ) << " <= " << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= " << AT( K(), "keys+1" ) << " then\n" + " " << GET_WIDE_KEY() << " - " << AT(K(), "keys" ) << " else slen)"; + out << + " let keys = " << vCS() << " lsl 1 in\n" + " let inds = " << AT( IO(), vCS() ) << " in\n" + "\n" + " let slen = " << AT( SP(), vCS() ) << " in\n" + " state.trans <- " << AT( I(), temp.str() ) << ";\n" + "\n"; +} + +void OCamlFlatCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "begin " << vCS() << " <- " << gotoDest << "; " << + CTRL_FLOW() << "raise Goto_again end"; +} + +void OCamlFlatCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "begin " << vCS() << " <- ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << " raise Goto_again end"; +} + +void OCamlFlatCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void OCamlFlatCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void OCamlFlatCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " <- " << nextDest << ";"; +} + +void OCamlFlatCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " <- ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void OCamlFlatCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "begin "; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "begin " << AT( STACK(), POST_INCR(TOP()) ) << " <- " << vCS() << "; "; + ret << vCS() << " <- " << callDest << "; " << CTRL_FLOW() << "raise Goto_again end "; + + if ( prePushExpr != 0 ) + ret << "end"; +} + +void OCamlFlatCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "begin "; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "begin " << AT(STACK(), POST_INCR(TOP()) ) << " <- " << vCS() << "; " << vCS() << " <- ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); " << CTRL_FLOW() << "raise Goto_again end "; + + if ( prePushExpr != 0 ) + ret << "end"; +} + +void OCamlFlatCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "begin " << vCS() << " <- " << AT(STACK(), PRE_DECR(TOP()) ) << "; "; + + if ( postPopExpr != 0 ) { + ret << "begin "; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "end "; + } + + ret << CTRL_FLOW() << "raise Goto_again end"; +} + +void OCamlFlatCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + ret << "begin " << P() << " <- " << P() << " + 1; " << CTRL_FLOW() << "raise Goto_out end"; +} + +void OCamlFlatCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); + + out << "type " << TYPE_STATE() << " = { mutable trans : int; mutable acts : int; mutable nacts : int; }" + << TOP_SEP(); + + out << "exception Goto_match" << TOP_SEP(); + out << "exception Goto_again" << TOP_SEP(); + out << "exception Goto_eof_trans" << TOP_SEP(); +} + +void OCamlFlatCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n"; + + out << + " _keys = " << vCS() << "<<1;\n" + " _conds = " << CO() << "[" << vCS() << "];\n" +// " _keys = " << ARR_OFF( CK(), "(" + vCS() + "<<1)" ) << ";\n" +// " _conds = " << ARR_OFF( C(), CO() + "[" + vCS() + "]" ) << ";\n" + "\n" + " _slen = " << CSP() << "[" << vCS() << "];\n" + " if (_slen > 0 && " << CK() << "[_keys] <=" + << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= " << CK() << "[_keys+1])\n" + " _cond = " << C() << "[_conds+" << GET_WIDE_KEY() << " - " << + CK() << "[_keys]];\n" + " else\n" + " _cond = 0;" + "\n"; + /* XXX This version of the code doesn't work because Mono is weird. Works + * fine in Microsoft's csc, even though the bug report filed claimed it + * didn't. + " _slen = " << CSP() << "[" << vCS() << "];\n" + " _cond = _slen > 0 && " << CK() << "[_keys] <=" + << GET_WIDE_KEY() << " &&\n" + " " << GET_WIDE_KEY() << " <= " << CK() << "[_keys+1] ?\n" + " " << C() << "[_conds+" << GET_WIDE_KEY() << " - " << CK() + << "[_keys]] : 0;\n" + "\n"; + */ + out << + " switch ( _cond ) {\n"; + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId + 1 << ": {\n"; + out << TABS(2) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << " }\n"; + out << " break;\n"; + } + + SWITCH_DEFAULT(); + + out << + " }\n"; +} + +void OCamlFlatCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " begin\n"; +// " " << slenType << " _slen"; + +// if ( redFsm->anyRegCurStateRef() ) +// out << ", _ps"; + +// out << +// " " << transType << " _trans"; + +// if ( redFsm->anyConditions() ) +// out << ", _cond"; +// out << ";\n"; + +// if ( redFsm->anyToStateActions() || +// redFsm->anyRegActions() || redFsm->anyFromStateActions() ) +// { +// out << +// " int _acts;\n" +// " int _nacts;\n"; +// } + +// out << +// " " << "int _keys;\n" +// " " << indsType << " _inds;\n"; + /* + " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" + " " << PTR_CONST() << ARRAY_TYPE(redFsm->maxIndex) << POINTER() << "_inds;\n";*/ + + if ( redFsm->anyConditions() ) { + out << + " " << condsType << " _conds;\n" + " " << WIDE_ALPH_TYPE() << " _widec;\n"; + } + + out << "\n"; + + out << + " let state = { trans = 0; acts = 0; nacts = 0; } in\n" + " let rec do_start () =\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " = " << PE() << " then\n" + " do_test_eof ()\n" + "\telse\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " = " << redFsm->errState->id << " then\n" + " do_out ()\n" + "\telse\n"; + } + + out << "\tdo_resume ()\n"; + + out << "and do_resume () =\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " state.acts <- " << AT( FSA(), vCS() ) << ";\n" + " state.nacts <- " << AT( A(), POST_INCR("state.acts") ) << ";\n" + " while " << POST_DECR("state.nacts") << " > 0 do\n" + " begin match " << AT( A(), POST_INCR("state.acts") ) << " with\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end\n" + " done;\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + +// out << "\tbegin try\n"; + LOCATE_TRANS(); +// out << "\twith Goto_match -> () end;\n"; + + out << "\tdo_eof_trans ()\n"; + +// if ( redFsm->anyEofTrans() ) + out << "and do_eof_trans () =\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " let ps = " << vCS() << " in\n"; + + out << + " " << vCS() << " <- " << AT( TT() ,"state.trans" ) << ";\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + "\tbegin try\n" + " match " << AT( TA(), "state.trans" ) << " with\n" + "\t| 0 -> raise Goto_again\n" + "\t| _ ->\n" + " state.acts <- " << AT( TA(), "state.trans" ) << ";\n" + " state.nacts <- " << AT( A(), POST_INCR("state.acts") ) << ";\n" + " while " << POST_DECR("state.nacts") << " > 0 do\n" + " begin match " << AT( A(), POST_INCR("state.acts") ) << " with\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + " done\n" + "\twith Goto_again -> () end;\n"; + } + out << "\tdo_again ()\n"; + +// if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || +// redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\tand do_again () =\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " state.acts <- " << AT( TSA(), vCS() ) << ";\n" + " state.nacts <- " << AT( A(), POST_INCR("state.acts") ) << ";\n" + " while " << POST_DECR("state.nacts") << " > 0 do\n" + " begin match " << AT( A(), POST_INCR("state.acts") ) << " with\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end\n" + " done;\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " match " << vCS() << " with\n" + "\t| " << redFsm->errState->id << " -> do_out ()\n" + "\t| _ ->\n"; + } + + out << "\t" << P() << " <- " << P() << " + 1;\n"; + + if ( !noEnd ) { + out << + " if " << P() << " <> " << PE() << " then\n" + " do_resume ()\n" + "\telse do_test_eof ()\n"; + } + else { + out << + " do_resume ()\n"; + } + +// if ( testEofUsed ) + out << "and do_test_eof () =\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " = " << vEOF() << " then\n" + " begin try\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << AT( ET(), vCS() ) << " > 0 then\n" + " begin\n" + " state.trans <- " << CAST(transType) << "(" << AT( ET(), vCS() ) << " - 1);\n" + " raise Goto_eof_trans;\n" + " end;\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " let __acts = ref " << AT( EA(), vCS() ) << " in\n" + " let __nacts = ref " << AT( A(), "!__acts" ) << " in\n" + " incr __acts;\n" + " while !__nacts > 0 do\n" + " decr __nacts;\n" + " begin match " << AT( A(), POST_INCR("__acts.contents") ) << " with\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + " done\n"; + } + + out << + " with Goto_again -> do_again ()\n" + " | Goto_eof_trans -> do_eof_trans () end\n" + "\n"; + } + else + { + out << "\t()\n"; + } + + if ( outLabelUsed ) + out << " and do_out () = ()\n"; + + out << "\tin do_start ()\n"; + out << " end;\n"; +} + +void OCamlFlatCodeGen::initVarTypes() +{ + slenType = ARRAY_TYPE(MAX(redFsm->maxSpan, redFsm->maxCondSpan)); + transType = ARRAY_TYPE(redFsm->maxIndex+1); + indsType = ARRAY_TYPE(redFsm->maxFlatIndexOffset); + condsType = ARRAY_TYPE(redFsm->maxCondIndexOffset); +} diff --git a/contrib/tools/ragel6/mlflat.h b/contrib/tools/ragel6/mlflat.h new file mode 100644 index 0000000000..6da18198d8 --- /dev/null +++ b/contrib/tools/ragel6/mlflat.h @@ -0,0 +1,91 @@ +/* + * Copyright 2004-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _MLFLAT_H +#define _MLFLAT_H + +#include <iostream> +#include "mlcodegen.h" + +/* Forwards. */ +//struct CodeGenData; +//struct NameInst; +//struct RedTransAp; +//struct RedStateAp; + +/* + * OCamlFlatCodeGen + */ +class OCamlFlatCodeGen : public OCamlCodeGen +{ +public: + OCamlFlatCodeGen( ostream &out ) : OCamlCodeGen(out) {} + virtual ~OCamlFlatCodeGen() { } + +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &FLAT_INDEX_OFFSET(); + std::ostream &KEY_SPANS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + void LOCATE_TRANS(); + + std::ostream &COND_INDEX_OFFSET(); + void COND_TRANSLATE(); + std::ostream &CONDS(); + std::ostream &COND_KEYS(); + std::ostream &COND_KEY_SPANS(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); + + void initVarTypes(); + string slenType, transType, indsType, condsType; +}; + +#endif diff --git a/contrib/tools/ragel6/mlftable.cpp b/contrib/tools/ragel6/mlftable.cpp new file mode 100644 index 0000000000..15a9885eec --- /dev/null +++ b/contrib/tools/ragel6/mlftable.cpp @@ -0,0 +1,462 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "mlftable.h" +#include "redfsm.h" +#include "gendata.h" + +/* Determine if we should use indicies or not. */ +void OCamlFTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActListId) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActListId) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &OCamlFTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &OCamlFTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + out << act; + return out; +} + +std::ostream &OCamlFTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + out << act; + return out; +} + + +/* Write out the function for a transition. */ +std::ostream &OCamlFTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + out << action; + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &OCamlFTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\t| " << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &OCamlFTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\t| " << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlFTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\t| " << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &OCamlFTabCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\t| " << redAct->actListId+1 << " ->\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +void OCamlFTabCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); + + out << "type " << TYPE_STATE() << " = { mutable keys : int; mutable trans : int; }" + << TOP_SEP(); + + out << "exception Goto_match" << TOP_SEP(); + out << "exception Goto_again" << TOP_SEP(); + out << "exception Goto_eof_trans" << TOP_SEP(); +} + +void OCamlFTabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " begin\n"; + +// if ( redFsm->anyRegCurStateRef() ) +// out << klenType ", _ps"; + + out << + " let state = { keys = 0; trans = 0; } in\n" + " let rec do_start () =\n"; + +// if ( redFsm->anyConditions() ) +// out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " = " << PE() << " then\n" + " do_test_eof ()\n" + "\telse\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " = " << redFsm->errState->id << " then\n" + " do_out ()\n" + "\telse\n"; + } + out << "\tdo_resume ()\n"; + + out << "and do_resume () =\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " begin match " << AT( FSA(), vCS() ) << " with\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + out << "\tbegin try\n"; + LOCATE_TRANS(); + out << "\twith Goto_match -> () end;\n"; + + out << + "\tdo_match ()\n"; + + out << "and do_match () =\n"; + + if ( useIndicies ) + out << " state.trans <- " << CAST(transType) << AT( I(), "state.trans" ) << ";\n"; + + out << "\tdo_eof_trans ()\n"; + +// if ( redFsm->anyEofTrans() ) + out << "and do_eof_trans () =\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " let ps = " << vCS() << " in\n"; + + out << + " " << vCS() << " <- " << AT( TT() ,"state.trans" ) << ";\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " begin try if " << AT( TA() , "state.trans" ) << " = 0 then\n" + " raise Goto_again;\n" + "\n" + " match " << AT( TA(), "state.trans" ) << " with\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " with Goto_again -> () end;\n" + "\n"; + } + out << "\tdo_again ()\n"; + +// if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || +// redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\tand do_again () =\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " begin match " << AT( TSA(), vCS() ) << " with\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " match " << vCS() << " with\n" + "\t| " << redFsm->errState->id << " -> do_out ()\n" + "\t| _ ->\n"; + } + + out << "\t" << P() << " <- " << P() << " + 1;\n"; + + if ( !noEnd ) { + out << + " if " << P() << " <> " << PE() << " then\n" + " do_resume ()\n" + "\telse do_test_eof ()\n"; + } + else { + out << + " do_resume ()\n"; + } + +// if ( testEofUsed ) + out << "and do_test_eof () =\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " = " << vEOF() << " then\n" + " begin try\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << AT( ET(), vCS() ) << " > 0 then\n" + " begin\n" + " state.trans <- " << CAST(transType) << "(" << AT( ET(), vCS() ) << " - 1);\n" + " raise Goto_eof_trans;\n" + " end;\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " begin match " << AT( EA(), vCS() ) << " with\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end\n"; + } + + out << + " with Goto_again -> do_again ()\n" + " | Goto_eof_trans -> do_eof_trans () end\n" + "\n"; + } + else + { + out << "\t()\n"; + } + + if ( outLabelUsed ) + out << " and do_out () = ()\n"; + + out << "\tin do_start ()\n"; + out << " end;\n"; +} + diff --git a/contrib/tools/ragel6/mlftable.h b/contrib/tools/ragel6/mlftable.h new file mode 100644 index 0000000000..da88e849fe --- /dev/null +++ b/contrib/tools/ragel6/mlftable.h @@ -0,0 +1,56 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _MLFTABLE_H +#define _MLFTABLE_H + +#include <iostream> +#include "mltable.h" + +/* Forwards. */ +//struct CodeGenData; + + +/* + * OCamlFTabCodeGen + */ +class OCamlFTabCodeGen : public OCamlTabCodeGen +{ +public: + OCamlFTabCodeGen( ostream &out ) : OCamlTabCodeGen(out) {} +private: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void writeData(); + virtual void writeExec(); + virtual void calcIndexSize(); +}; + +#endif diff --git a/contrib/tools/ragel6/mlgoto.cpp b/contrib/tools/ragel6/mlgoto.cpp new file mode 100644 index 0000000000..65570d8d86 --- /dev/null +++ b/contrib/tools/ragel6/mlgoto.cpp @@ -0,0 +1,821 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "mlgoto.h" +#include "redfsm.h" +#include "bstmap.h" +#include "gendata.h" + +/* Emit the goto to take for a given transition. */ +std::ostream &OCamlGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + out << TABS(level) << "tr" << trans->id << " ()"; + return out; +} + +std::ostream &OCamlGotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, false ); + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlGotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, false ); + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlGotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, true ); + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlGotoCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, false ); + out << "\t()\n"; + } + } + + genLineDirective( out ); + return out; +} + +void OCamlGotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + /* Label the state. */ + out << "| " << state->id << " ->\n"; +} + + +void OCamlGotoCodeGen::emitSingleSwitch( RedStateAp *state ) +{ + /* Load up the singles. */ + int numSingles = state->outSingle.length(); + RedTransEl *data = state->outSingle.data; + + if ( numSingles == 1 ) { + /* If there is a single single key then write it out as an if. */ + out << "\tif " << GET_WIDE_KEY(state) << " = " << + KEY(data[0].lowKey) << " then\n\t\t"; + + /* Virtual function for writing the target of the transition. */ + TRANS_GOTO(data[0].value, 0) << " else\n"; + } + else if ( numSingles > 1 ) { + /* Write out single keys in a switch if there is more than one. */ + out << "\tmatch " << GET_WIDE_KEY(state) << " with\n"; + + /* Write out the single indicies. */ + for ( int j = 0; j < numSingles; j++ ) { + out << "\t\t| " << ALPHA_KEY(data[j].lowKey) << " -> "; + TRANS_GOTO(data[j].value, 0) << "\n"; + } + + out << "\t\t| _ ->\n"; + } +} + +void OCamlGotoCodeGen::emitRangeBSearch( RedStateAp *state, int level, int low, int high, RedTransAp* def) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + RedTransEl *data = state->outRange.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid].lowKey == keyOps->minKey; + bool limitHigh = data[mid].highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " then begin\n"; + emitRangeBSearch( state, level+1, low, mid-1, def ); + out << TABS(level) << " end else if " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " then begin\n"; + emitRangeBSearch( state, level+1, mid+1, high, def ); + out << TABS(level) << " end else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " then begin\n"; + emitRangeBSearch( state, level+1, low, mid-1, def ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << " end else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << " end else if " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " then\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n" << TABS(level) << "else\n"; + TRANS_GOTO(def, level+1) << "\n"; + } + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " then begin\n"; + emitRangeBSearch( state, level+1, mid+1, high, def ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << " end else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << " end else if " << GET_WIDE_KEY(state) << " >= " << + KEY(data[mid].lowKey) << " then\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n" << TABS(level) << "else\n"; + TRANS_GOTO(def, level+1) << "\n"; + } + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " && " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " then\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n" << TABS(level) << "else\n"; + TRANS_GOTO(def, level+1) << "\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " then\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n" << TABS(level) << "else\n"; + TRANS_GOTO(def, level+1) << "\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " then\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n" << TABS(level) << "else\n"; + TRANS_GOTO(def, level+1) << "\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } +} + +void OCamlGotoCodeGen::STATE_GOTO_ERROR() +{ + /* Label the state and bail immediately. */ + outLabelUsed = true; + RedStateAp *state = redFsm->errState; + out << "| " << state->id << " ->\n"; + out << " do_out ()\n"; +} + +void OCamlGotoCodeGen::COND_TRANSLATE( GenStateCond *stateCond, int level ) +{ + GenCondSpace *condSpace = stateCond->condSpace; + out << TABS(level) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(level) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } +} + +void OCamlGotoCodeGen::emitCondBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + GenStateCond **data = state->stateCondVect.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid]->lowKey == keyOps->minKey; + bool limitHigh = data[mid]->highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if ( " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " ) {\n"; + emitCondBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "} else if ( " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " ) {\n"; + emitCondBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if ( " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " ) {\n"; + emitCondBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if ( " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " ) {\n"; + emitCondBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "} else {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + out << TABS(level) << "} else if ( " << GET_KEY() << " >= " << + KEY(data[mid]->lowKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " && " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if ( " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " ) {\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if ( " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " )\n {"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "}\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + COND_TRANSLATE(data[mid], level); + } + } +} + +std::ostream &OCamlGotoCodeGen::STATE_GOTOS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* Writing code above state gotos. */ + GOTO_HEADER( st ); + out << "\tbegin\n"; + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1, st->defTrans ); + else + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + + out << "\tend\n"; + } + } + return out; +} + +std::ostream &OCamlGotoCodeGen::TRANSITIONS() +{ + /* Emit any transitions that have functions and that go to + * this state. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Write the label for the transition so it can be jumped to. */ + out << " and tr" << trans->id << " () = "; + + /* Destination state. */ + if ( trans->action != 0 && trans->action->anyCurStateRef() ) + out << "_ps = " << vCS() << ";"; + out << vCS() << " <- " << trans->targ->id << "; "; + + if ( trans->action != 0 ) { + /* Write out the transition func. */ + out << "f" << trans->action->actListId << " ()\n"; + } + else { + /* No code to execute, just loop around. */ + out << "do_again ()\n"; + } + } + return out; +} + +std::ostream &OCamlGotoCodeGen::EXEC_FUNCS() +{ + /* Make labels that set acts and jump to execFuncs. Loop func indicies. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + out << " and f" << redAct->actListId << " () = " << + "state.acts <- " << itoa( redAct->location+1 ) << "; " + "execFuncs ()\n"; + } + } + + out << + "\n" + "and execFuncs () =\n" + " state.nacts <- " << AT( A(), POST_INCR( "state.acts") ) << ";\n" + " begin try while " << POST_DECR("state.nacts") << " > 0 do\n" + " match " << AT( A(), POST_INCR("state.acts") ) << " with\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " done with Goto_again -> () end;\n" + " do_again ()\n"; + return out; +} + +unsigned int OCamlGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +unsigned int OCamlGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +unsigned int OCamlGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + +std::ostream &OCamlGotoCodeGen::TO_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = TO_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ARR_SEP(); + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &OCamlGotoCodeGen::FROM_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = FROM_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ARR_SEP(); + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &OCamlGotoCodeGen::EOF_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = EOF_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ARR_SEP(); + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &OCamlGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\t| " << st->id << " -> "; + + /* Write the goto func. */ + out << "f" << st->eofAction->actListId << " ()\n"; + } + } + + return out; +} + +void OCamlGotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "begin " << vCS() << " <- " << gotoDest << "; " << + CTRL_FLOW() << "raise Goto_again end"; +} + +void OCamlGotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "begin " << vCS() << " <- ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "raise Goto_again end"; +} + +void OCamlGotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void OCamlGotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void OCamlGotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " <- " << nextDest << ";"; +} + +void OCamlGotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " <- ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void OCamlGotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "begin "; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "begin " << AT( STACK(), POST_INCR(TOP()) ) << " <- " << vCS() << "; "; + ret << vCS() << " <- " << callDest << "; " << CTRL_FLOW() << "raise Goto_again end "; + + if ( prePushExpr != 0 ) + ret << "end"; +} + +void OCamlGotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "begin "; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "begin " << AT(STACK(), POST_INCR(TOP()) ) << " <- " << vCS() << "; " << vCS() << " <- ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); " << CTRL_FLOW() << "raise Goto_again end "; + + if ( prePushExpr != 0 ) + ret << "end"; +} + +void OCamlGotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "begin " << vCS() << " <- " << AT(STACK(), PRE_DECR(TOP()) ) << "; "; + + if ( postPopExpr != 0 ) { + ret << "begin "; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "end "; + } + + ret << CTRL_FLOW() << "raise Goto_again end"; +} + +void OCamlGotoCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + ret << "begin " << P() << " <- " << P() << " + 1; " << CTRL_FLOW() << "raise Goto_out end"; +} + +void OCamlGotoCodeGen::writeData() +{ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); + + out << "type " << TYPE_STATE() << " = { mutable acts : " << ARRAY_TYPE(redFsm->maxActionLoc) << + " ; mutable nacts : " << ARRAY_TYPE(redFsm->maxActArrItem) << "; }" + << TOP_SEP(); + + out << "exception Goto_again" << TOP_SEP(); +} + +void OCamlGotoCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + + out << " begin\n"; + +// if ( redFsm->anyRegCurStateRef() ) +// out << " int _ps = 0;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << " let state = { acts = 0; nacts = 0; } in\n"; + } + +// if ( redFsm->anyConditions() ) +// out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + out << "\n"; + out << " let rec do_start () =\n"; + + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " = " << PE() << " then\n" + " do_test_eof ()\n" + "\telse\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " = " << redFsm->errState->id << " then\n" + " do_out ()\n" + "\telse\n"; + } + out << "\tdo_resume ()\n"; + + out << "and do_resume () =\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " state.acts <- " << AT( FSA(), vCS() ) << ";\n" + " state.nacts <- " << AT( A(), POST_INCR("state.acts") ) << ";\n" + " while " << POST_DECR("state.nacts") << " > 0 do\n" + " begin match " << AT( A(), POST_INCR("state.acts") ) << " with\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end\n" + " done;\n" + "\n"; + } + + out << + " begin match " << vCS() << " with\n"; + STATE_GOTOS(); + SWITCH_DEFAULT() << + " end\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_FUNCS() << "\n"; + +// if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || +// redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\tand do_again () =\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " state.acts <- " << AT( TSA(), vCS() ) << ";\n" + " state.nacts <- " << AT( A(), POST_INCR("state.acts") ) << ";\n" + " while " << POST_DECR("state.nacts") << " > 0 do\n" + " begin match " << AT( A(), POST_INCR("state.acts") ) << " with\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end\n" + " done;\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " match " << vCS() << " with\n" + "\t| " << redFsm->errState->id << " -> do_out ()\n" + "\t| _ ->\n"; + } + + out << "\t" << P() << " <- " << P() << " + 1;\n"; + + if ( !noEnd ) { + out << + " if " << P() << " <> " << PE() << " then\n" + " do_resume ()\n" + "\telse do_test_eof ()\n"; + } + else { + out << + " do_resume ()\n"; + } + +// if ( testEofUsed ) + out << "and do_test_eof () =\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " = " << vEOF() << " then\n" + " begin\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " match " << vCS() << " with\n"; + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) + out << " | " << st->id << " -> tr" << st->eofTrans->id << " ()\n"; + } + + out << "\t| _ -> ();\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " let __acts = ref " << AT( EA(), vCS() ) << " in\n" + " let __nacts = ref " << AT( A(), "!__acts" ) << " in\n" + " incr __acts;\n" + " begin try while !__nacts > 0 do\n" + " decr __nacts;\n" + " begin match " << AT( A(), POST_INCR("__acts.contents") ) << " with\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + " done with Goto_again -> do_again () end;\n"; + } + + out << + " end\n" + "\n"; + } + else + { + out << "\t()\n"; + } + + if ( outLabelUsed ) + out << " and do_out () = ()\n"; + + out << "\tin do_start ()\n"; + out << " end;\n"; +} diff --git a/contrib/tools/ragel6/mlgoto.h b/contrib/tools/ragel6/mlgoto.h new file mode 100644 index 0000000000..50aeb32a2f --- /dev/null +++ b/contrib/tools/ragel6/mlgoto.h @@ -0,0 +1,89 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _MLGOTO_H +#define _MLGOTO_H + +#include <iostream> +#include "mlcodegen.h" + +/* Forwards. */ +//struct CodeGenData; +//struct NameInst; +//struct RedTransAp; +//struct RedStateAp; +//struct GenStateCond; + +/* + * OCamlGotoCodeGen + */ +class OCamlGotoCodeGen : virtual public OCamlCodeGen +{ +public: + OCamlGotoCodeGen( ostream &out ) : OCamlCodeGen(out) {} + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &STATE_GOTOS(); + std::ostream &TRANSITIONS(); + std::ostream &EXEC_FUNCS(); + std::ostream &FINISH_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + virtual unsigned int TO_STATE_ACTION( RedStateAp *state ); + virtual unsigned int FROM_STATE_ACTION( RedStateAp *state ); + virtual unsigned int EOF_ACTION( RedStateAp *state ); + + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + + void COND_TRANSLATE( GenStateCond *stateCond, int level ); + void emitCondBSearch( RedStateAp *state, int level, int low, int high ); + void STATE_CONDS( RedStateAp *state, bool genDefault ); + + virtual std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + + void emitSingleSwitch( RedStateAp *state ); + void emitRangeBSearch( RedStateAp *state, int level, int low, int high, RedTransAp* def ); + + /* Called from STATE_GOTOS just before writing the gotos */ + virtual void GOTO_HEADER( RedStateAp *state ); + virtual void STATE_GOTO_ERROR(); + + virtual void writeData(); + virtual void writeExec(); +}; + +#endif diff --git a/contrib/tools/ragel6/mltable.cpp b/contrib/tools/ragel6/mltable.cpp new file mode 100644 index 0000000000..8f70b7cc77 --- /dev/null +++ b/contrib/tools/ragel6/mltable.cpp @@ -0,0 +1,1131 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "ragel.h" +#include "mltable.h" +#include "redfsm.h" +#include "gendata.h" + +/* Determine if we should use indicies or not. */ +void OCamlTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActionLoc) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActionLoc) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +std::ostream &OCamlTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + out << act; + return out; +} + +std::ostream &OCamlTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + out << act; + return out; +} + +std::ostream &OCamlTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + out << act; + return out; +} + + +std::ostream &OCamlTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + out << act; + return out; +} + +std::ostream &OCamlTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\t| " << act->actionId << " ->\n"; + ACTION( out, act, 0, true ); + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &OCamlTabCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\t| " << act->actionId << " -> \n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &OCamlTabCodeGen::COND_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->stateCondList.length(); + } + out << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::KEY_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + out << curKeyOffset; + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the key offset ahead. */ + curKeyOffset += st->outSingle.length() + st->outRange.length()*2; + } + out << "\n"; + return out; +} + + +std::ostream &OCamlTabCodeGen::INDEX_OFFSETS() +{ + out << "\t"; + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + out << curIndOffset; + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + + /* Move the index offset ahead. */ + curIndOffset += st->outSingle.length() + st->outRange.length(); + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + out << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::COND_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->stateCondList.length(); + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &OCamlTabCodeGen::SINGLE_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + out << st->outSingle.length(); + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::RANGE_LENS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit length of range index. */ + out << st->outRange.length(); + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::TO_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + TO_STATE_ACTION(st); + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::FROM_STATE_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + FROM_STATE_ACTION(st); + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::EOF_ACTIONS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + EOF_ACTION(st); + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::EOF_TRANS() +{ + out << "\t"; + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + out << trans; + + if ( !st.last() ) { + out << ARR_SEP(); + if ( ++totalStateNum % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + return out; +} + + +std::ostream &OCamlTabCodeGen::COND_KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Lower key. */ + out << ALPHA_KEY( sc->lowKey ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + /* Upper key. */ + out << ALPHA_KEY( sc->highKey ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::COND_SPACES() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Cond Space id. */ + out << sc->condSpace->condSpaceId << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::KEYS() +{ + out << '\t'; + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << ALPHA_KEY( stel->lowKey ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Loop the state's transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + /* Lower key. */ + out << ALPHA_KEY( rtel->lowKey ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + /* Upper key. */ + out << ALPHA_KEY( rtel->highKey ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::INDICIES() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + out << stel->value->id << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + out << rtel->value->id << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + out << st->defTrans->id << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::TRANS_TARGS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + out << trans->targ->id << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + out << trans->targ->id << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default target state. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + out << trans->targ->id << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + trans->pos = totalTrans; + out << trans->targ->id << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + + +std::ostream &OCamlTabCodeGen::TRANS_ACTIONS() +{ + int totalTrans = 0; + out << '\t'; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + TRANS_ACTION( trans ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + TRANS_ACTION( trans ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + TRANS_ACTION( trans ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + TRANS_ACTION( trans ) << ARR_SEP(); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + out << 0 << "\n"; + return out; +} + +std::ostream &OCamlTabCodeGen::TRANS_TARGS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Record the position, need this for eofTrans. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + out << trans->targ->id; + if ( t < redFsm->transSet.length()-1 ) { + out << ARR_SEP(); + if ( ++totalStates % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + + +std::ostream &OCamlTabCodeGen::TRANS_ACTIONS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + out << '\t'; + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + TRANS_ACTION( trans ); + if ( t < redFsm->transSet.length()-1 ) { + out << ARR_SEP(); + if ( ++totalAct % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] transPtrs; + return out; +} + +void OCamlTabCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "begin " << vCS() << " <- " << gotoDest << "; " << + CTRL_FLOW() << "raise Goto_again end"; +} + +void OCamlTabCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "begin " << vCS() << " <- ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << "); " << CTRL_FLOW() << "raise Goto_again end"; +} + +void OCamlTabCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void OCamlTabCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void OCamlTabCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " <- " << nextDest << ";"; +} + +void OCamlTabCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " <- ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void OCamlTabCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "begin "; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "begin " << AT( STACK(), POST_INCR(TOP()) ) << " <- " << vCS() << "; "; + ret << vCS() << " <- " << callDest << "; " << CTRL_FLOW() << "raise Goto_again end "; + + if ( prePushExpr != 0 ) + ret << "end"; +} + +void OCamlTabCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "begin "; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "begin " << AT(STACK(), POST_INCR(TOP()) ) << " <- " << vCS() << "; " << vCS() << " <- ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); " << CTRL_FLOW() << "raise Goto_again end "; + + if ( prePushExpr != 0 ) + ret << "end"; +} + +void OCamlTabCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "begin " << vCS() << " <- " << AT(STACK(), PRE_DECR(TOP()) ) << "; "; + + if ( postPopExpr != 0 ) { + ret << "begin "; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "end "; + } + + ret << CTRL_FLOW() << "raise Goto_again end"; +} + +void OCamlTabCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + ret << "begin " << P() << " <- " << P() << " + 1; " << CTRL_FLOW() << "raise Goto_out end"; +} + +void OCamlTabCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); + + out << "type " << TYPE_STATE() << " = { mutable keys : int; mutable trans : int; mutable acts : int; mutable nacts : int; }" + << TOP_SEP(); + + out << "exception Goto_match" << TOP_SEP(); + out << "exception Goto_again" << TOP_SEP(); + out << "exception Goto_eof_trans" << TOP_SEP(); +} + +void OCamlTabCodeGen::LOCATE_TRANS() +{ + out << + " state.keys <- " << AT( KO(), vCS() ) << ";\n" + " state.trans <- " << CAST(transType) << AT( IO(), vCS() ) << ";\n" + "\n" + " let klen = " << AT( SL(), vCS() ) << " in\n" + " if klen > 0 then begin\n" + " let lower : " << signedKeysType << " ref = ref state.keys in\n" + " let upper : " << signedKeysType << " ref = ref " << CAST(signedKeysType) << + "(state.keys + klen - 1) in\n" + " while !upper >= !lower do\n" + " let mid = " << CAST(signedKeysType) << " (!lower + ((!upper - !lower) / 2)) in\n" + " if " << GET_WIDE_KEY() << " < " << AT( K(), "mid" ) << " then\n" + " upper := " << CAST(signedKeysType) << " (mid - 1)\n" + " else if " << GET_WIDE_KEY() << " > " << AT( K(), "mid" ) << " then\n" + " lower := " << CAST(signedKeysType) << " (mid + 1)\n" + " else begin\n" + " state.trans <- state.trans + " << CAST(transType) << " (mid - state.keys);\n" + " raise Goto_match;\n" + " end\n" + " done;\n" + " state.keys <- state.keys + " << CAST(keysType) << " klen;\n" + " state.trans <- state.trans + " << CAST(transType) << " klen;\n" + " end;\n" + "\n" + " let klen = " << AT( RL(), vCS() ) << " in\n" + " if klen > 0 then begin\n" + " let lower : " << signedKeysType << " ref = ref state.keys in\n" + " let upper : " << signedKeysType << " ref = ref " << CAST(signedKeysType) << + "(state.keys + (klen * 2) - 2) in\n" + " while !upper >= !lower do\n" + " let mid = " << CAST(signedKeysType) << " (!lower + (((!upper - !lower) / 2) land (lnot 1))) in\n" + " if " << GET_WIDE_KEY() << " < " << AT( K() , "mid" ) << " then\n" + " upper := " << CAST(signedKeysType) << " (mid - 2)\n" + " else if " << GET_WIDE_KEY() << " > " << AT( K(), "mid+1" ) << " then\n" + " lower := " << CAST(signedKeysType) << " (mid + 2)\n" + " else begin\n" + " state.trans <- state.trans + " << CAST(transType) << "((mid - state.keys) / 2);\n" + " raise Goto_match;\n" + " end\n" + " done;\n" + " state.trans <- state.trans + " << CAST(transType) << " klen;\n" + " end;\n" + "\n"; +} + +void OCamlTabCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << ";\n" + " _klen = " << CL() << "[" << vCS() << "];\n" + " _keys = " << CAST(keysType) << " ("<< CO() << "[" << vCS() << "]*2);\n" + " if ( _klen > 0 ) {\n" + " " << signedKeysType << " _lower = _keys;\n" + " " << signedKeysType << " _mid;\n" + " " << signedKeysType << " _upper = " << CAST(signedKeysType) << + " (_keys + (_klen<<1) - 2);\n" + " while (true) {\n" + " if ( _upper < _lower )\n" + " break;\n" + "\n" + " _mid = " << CAST(signedKeysType) << + " (_lower + (((_upper-_lower) >> 1) & ~1));\n" + " if ( " << GET_WIDE_KEY() << " < " << CK() << "[_mid] )\n" + " _upper = " << CAST(signedKeysType) << " (_mid - 2);\n" + " else if ( " << GET_WIDE_KEY() << " > " << CK() << "[_mid+1] )\n" + " _lower = " << CAST(signedKeysType) << " (_mid + 2);\n" + " else {\n" + " switch ( " << C() << "[" << CO() << "[" << vCS() << "]" + " + ((_mid - _keys)>>1)] ) {\n"; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " case " << condSpace->condSpaceId << ": {\n"; + out << TABS(2) << "_widec = " << CAST(WIDE_ALPH_TYPE()) << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "));\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " ) _widec += " << condValOffset << ";\n"; + } + + out << + " break;\n" + " }\n"; + } + + SWITCH_DEFAULT(); + + out << + " }\n" + " break;\n" + " }\n" + " }\n" + " }\n" + "\n"; +} + +void OCamlTabCodeGen::writeExec() +{ + testEofUsed = false; + outLabelUsed = false; + initVarTypes(); + + out << + " begin\n"; +// " " << klenType << " _klen"; + +// if ( redFsm->anyRegCurStateRef() ) +// out << ", _ps"; + +/* + out << " " << transType << " _trans;\n"; + + if ( redFsm->anyConditions() ) + out << " " << WIDE_ALPH_TYPE() << " _widec;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << + " int _acts;\n" + " int _nacts;\n"; + } + + out << + " " << keysType << " _keys;\n" + "\n"; +// " " << PTR_CONST() << WIDE_ALPH_TYPE() << POINTER() << "_keys;\n" +*/ + + out << + " let state = { keys = 0; trans = 0; acts = 0; nacts = 0; } in\n" + " let rec do_start () =\n"; + if ( !noEnd ) { + testEofUsed = true; + out << + " if " << P() << " = " << PE() << " then\n" + " do_test_eof ()\n" + "\telse\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if " << vCS() << " = " << redFsm->errState->id << " then\n" + " do_out ()\n" + "\telse\n"; + } + out << "\tdo_resume ()\n"; + + out << "and do_resume () =\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " state.acts <- " << AT( FSA(), vCS() ) << ";\n" + " state.nacts <- " << AT( A(), POST_INCR("state.acts") ) << ";\n" + " while " << POST_DECR("state.nacts") << " > 0 do\n" + " begin match " << AT( A(), POST_INCR("state.acts") ) << " with\n"; + FROM_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end\n" + " done;\n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + out << "\tbegin try\n"; + LOCATE_TRANS(); + out << "\twith Goto_match -> () end;\n"; + + out << + "\tdo_match ()\n"; + + out << "and do_match () =\n"; + + if ( useIndicies ) + out << " state.trans <- " << CAST(transType) << AT( I(), "state.trans" ) << ";\n"; + + out << "\tdo_eof_trans ()\n"; + +// if ( redFsm->anyEofTrans() ) + out << "and do_eof_trans () =\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " let ps = " << vCS() << " in\n"; + + out << + " " << vCS() << " <- " << AT( TT() ,"state.trans" ) << ";\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + "\tbegin try\n" + " match " << AT( TA(), "state.trans" ) << " with\n" + "\t| 0 -> raise Goto_again\n" + "\t| _ ->\n" + " state.acts <- " << AT( TA(), "state.trans" ) << ";\n" + " state.nacts <- " << AT( A(), POST_INCR("state.acts") ) << ";\n" + " while " << POST_DECR("state.nacts") << " > 0 do\n" + " begin match " << AT( A(), POST_INCR("state.acts") ) << " with\n"; + ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + " done\n" + "\twith Goto_again -> () end;\n"; + } + out << "\tdo_again ()\n"; + +// if ( redFsm->anyRegActions() || redFsm->anyActionGotos() || +// redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << "\tand do_again () =\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " state.acts <- " << AT( TSA(), vCS() ) << ";\n" + " state.nacts <- " << AT( A(), POST_INCR("state.acts") ) << ";\n" + " while " << POST_DECR("state.nacts") << " > 0 do\n" + " begin match " << AT( A(), POST_INCR("state.acts") ) << " with\n"; + TO_STATE_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end\n" + " done;\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " match " << vCS() << " with\n" + "\t| " << redFsm->errState->id << " -> do_out ()\n" + "\t| _ ->\n"; + } + + out << "\t" << P() << " <- " << P() << " + 1;\n"; + + if ( !noEnd ) { + out << + " if " << P() << " <> " << PE() << " then\n" + " do_resume ()\n" + "\telse do_test_eof ()\n"; + } + else { + out << + " do_resume ()\n"; + } + +// if ( testEofUsed ) + out << "and do_test_eof () =\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " = " << vEOF() << " then\n" + " begin try\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << AT( ET(), vCS() ) << " > 0 then\n" + " begin\n" + " state.trans <- " << CAST(transType) << "(" << AT( ET(), vCS() ) << " - 1);\n" + " raise Goto_eof_trans;\n" + " end;\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " let __acts = ref " << AT( EA(), vCS() ) << " in\n" + " let __nacts = ref " << AT( A(), "!__acts" ) << " in\n" + " incr __acts;\n" + " while !__nacts > 0 do\n" + " decr __nacts;\n" + " begin match " << AT( A(), POST_INCR("__acts.contents") ) << " with\n"; + EOF_ACTION_SWITCH(); + SWITCH_DEFAULT() << + " end;\n" + " done\n"; + } + + out << + " with Goto_again -> do_again ()\n" + " | Goto_eof_trans -> do_eof_trans () end\n" + "\n"; + } + else + { + out << "\t()\n"; + } + + if ( outLabelUsed ) + out << " and do_out () = ()\n"; + + out << "\tin do_start ()\n"; + out << " end;\n"; +} + +void OCamlTabCodeGen::initVarTypes() +{ + int klenMax = MAX(MAX(redFsm->maxCondLen, redFsm->maxRangeLen), + redFsm->maxSingleLen); + int keysMax = MAX(MAX(redFsm->maxKeyOffset, klenMax), + redFsm->maxCondOffset); + int transMax = MAX(MAX(redFsm->maxIndex+1, redFsm->maxIndexOffset), keysMax); + transMax = MAX(transMax, klenMax); + transType = ARRAY_TYPE(transMax); + klenType = ARRAY_TYPE(klenMax); + keysType = ARRAY_TYPE(keysMax); + signedKeysType = ARRAY_TYPE(keysMax, true); +} diff --git a/contrib/tools/ragel6/mltable.h b/contrib/tools/ragel6/mltable.h new file mode 100644 index 0000000000..505d378a89 --- /dev/null +++ b/contrib/tools/ragel6/mltable.h @@ -0,0 +1,102 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + * 2004 Erich Ocean <eric.ocean@ampede.com> + * 2005 Alan West <alan@alanz.com> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _OCAMLTABCODEGEN_H +#define _OCAMLTABCODEGEN_H + +#include <iostream> +#include "mlcodegen.h" + +/* Forwards. */ +/* +struct CodeGenData; +struct NameInst; +struct RedTransAp; +struct RedStateAp; +*/ + +/* + * OCamlTabCodeGen + */ +class OCamlTabCodeGen : public OCamlCodeGen +{ +public: + OCamlTabCodeGen( ostream &out ) : OCamlCodeGen(out) {} + virtual ~OCamlTabCodeGen() { } + virtual void writeData(); + virtual void writeExec(); + +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + std::ostream &COND_KEYS(); + std::ostream &COND_SPACES(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &COND_OFFSETS(); + std::ostream &KEY_OFFSETS(); + std::ostream &INDEX_OFFSETS(); + std::ostream &COND_LENS(); + std::ostream &SINGLE_LENS(); + std::ostream &RANGE_LENS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + std::ostream &TRANS_TARGS_WI(); + std::ostream &TRANS_ACTIONS_WI(); + + void LOCATE_TRANS(); + + void COND_TRANSLATE(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + virtual std::ostream &TO_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &FROM_STATE_ACTION( RedStateAp *state ); + virtual std::ostream &EOF_ACTION( RedStateAp *state ); + virtual std::ostream &TRANS_ACTION( RedTransAp *trans ); + virtual void calcIndexSize(); + + void initVarTypes(); + string klenType; + string keysType; + string signedKeysType; + string transType; +}; + +#endif diff --git a/contrib/tools/ragel6/parsedata.cpp b/contrib/tools/ragel6/parsedata.cpp new file mode 100644 index 0000000000..eafe73e524 --- /dev/null +++ b/contrib/tools/ragel6/parsedata.cpp @@ -0,0 +1,1491 @@ +/* + * Copyright 2001-2008 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iostream> +#include <iomanip> +#include <errno.h> +#include <stdlib.h> +#include <limits.h> + +#include "ragel.h" +#include "rlparse.h" +#include "parsedata.h" +#include "parsetree.h" +#include "mergesort.h" +#include "xmlcodegen.h" +#include "version.h" +#include "inputdata.h" + +using namespace std; + +char mainMachine[] = "main"; + +void Token::set( const char *str, int len ) +{ + length = len; + data = new char[len+1]; + memcpy( data, str, len ); + data[len] = 0; +} + +void Token::append( const Token &other ) +{ + int newLength = length + other.length; + char *newString = new char[newLength+1]; + memcpy( newString, data, length ); + memcpy( newString + length, other.data, other.length ); + newString[newLength] = 0; + data = newString; + length = newLength; +} + +/* Perform minimization after an operation according + * to the command line args. */ +void afterOpMinimize( FsmAp *fsm, bool lastInSeq ) +{ + /* Switch on the prefered minimization algorithm. */ + if ( minimizeOpt == MinimizeEveryOp || ( minimizeOpt == MinimizeMostOps && lastInSeq ) ) { + /* First clean up the graph. FsmAp operations may leave these + * lying around. There should be no dead end states. The subtract + * intersection operators are the only places where they may be + * created and those operators clean them up. */ + fsm->removeUnreachableStates(); + + switch ( minimizeLevel ) { + case MinimizeApprox: + fsm->minimizeApproximate(); + break; + case MinimizePartition1: + fsm->minimizePartition1(); + break; + case MinimizePartition2: + fsm->minimizePartition2(); + break; + case MinimizeStable: + fsm->minimizeStable(); + break; + } + } +} + +/* Count the transitions in the fsm by walking the state list. */ +int countTransitions( FsmAp *fsm ) +{ + int numTrans = 0; + StateAp *state = fsm->stateList.head; + while ( state != 0 ) { + numTrans += state->outList.length(); + state = state->next; + } + return numTrans; +} + +Key makeFsmKeyHex( char *str, const InputLoc &loc, ParseData *pd ) +{ + /* Reset errno so we can check for overflow or underflow. In the event of + * an error, sets the return val to the upper or lower bound being tested + * against. */ + errno = 0; + unsigned int size = keyOps->alphType->size; + bool unusedBits = size < sizeof(unsigned long); + + unsigned long ul = strtoul( str, 0, 16 ); + + if ( errno == ERANGE || ( unusedBits && ul >> (size * 8) ) ) { + error(loc) << "literal " << str << " overflows the alphabet type" << endl; + ul = 1 << (size * 8); + } + + if ( unusedBits && keyOps->alphType->isSigned && ul >> (size * 8 - 1) ) + ul |= ( (unsigned long)(-1L) >> (size*8) ) << (size*8); + + return Key( (long)ul ); +} + +#ifdef _MSC_VER +# define strtoll _strtoi64 +#endif + +Key makeFsmKeyDec( char *str, const InputLoc &loc, ParseData *pd ) +{ + if ( keyOps->alphType->isSigned ) { + /* Convert the number to a decimal. First reset errno so we can check + * for overflow or underflow. */ + errno = 0; + long long minVal = keyOps->alphType->sMinVal; + long long maxVal = keyOps->alphType->sMaxVal; + + long long ll = strtoll( str, 0, 10 ); + + /* Check for underflow. */ + if ( ( errno == ERANGE && ll < 0 ) || ll < minVal) { + error(loc) << "literal " << str << " underflows the alphabet type" << endl; + ll = minVal; + } + /* Check for overflow. */ + else if ( ( errno == ERANGE && ll > 0 ) || ll > maxVal ) { + error(loc) << "literal " << str << " overflows the alphabet type" << endl; + ll = maxVal; + } + + return Key( (long)ll ); + } + else { + /* Convert the number to a decimal. First reset errno so we can check + * for overflow or underflow. */ + errno = 0; + unsigned long long minVal = keyOps->alphType->uMinVal; + unsigned long long maxVal = keyOps->alphType->uMaxVal; + + unsigned long long ull = strtoull( str, 0, 10 ); + + /* Check for underflow. */ + if ( ( errno == ERANGE && ull < 0 ) || ull < minVal) { + error(loc) << "literal " << str << " underflows the alphabet type" << endl; + ull = minVal; + } + /* Check for overflow. */ + else if ( ( errno == ERANGE && ull > 0 ) || ull > maxVal ) { + error(loc) << "literal " << str << " overflows the alphabet type" << endl; + ull = maxVal; + } + + return Key( (unsigned long)ull ); + } +} + +/* Make an fsm key in int format (what the fsm graph uses) from an alphabet + * number returned by the parser. Validates that the number doesn't overflow + * the alphabet type. */ +Key makeFsmKeyNum( char *str, const InputLoc &loc, ParseData *pd ) +{ + /* Switch on hex/decimal format. */ + if ( str[0] == '0' && str[1] == 'x' ) + return makeFsmKeyHex( str, loc, pd ); + else + return makeFsmKeyDec( str, loc, pd ); +} + +/* Make an fsm int format (what the fsm graph uses) from a single character. + * Performs proper conversion depending on signed/unsigned property of the + * alphabet. */ +Key makeFsmKeyChar( char c, ParseData *pd ) +{ + if ( keyOps->isSigned ) { + /* Copy from a char type. */ + return Key( c ); + } + else { + /* Copy from an unsigned byte type. */ + return Key( (unsigned char)c ); + } +} + +/* Make an fsm key array in int format (what the fsm graph uses) from a string + * of characters. Performs proper conversion depending on signed/unsigned + * property of the alphabet. */ +void makeFsmKeyArray( Key *result, char *data, int len, ParseData *pd ) +{ + if ( keyOps->isSigned ) { + /* Copy from a char star type. */ + char *src = data; + for ( int i = 0; i < len; i++ ) + result[i] = Key(src[i]); + } + else { + /* Copy from an unsigned byte ptr type. */ + unsigned char *src = (unsigned char*) data; + for ( int i = 0; i < len; i++ ) + result[i] = Key(src[i]); + } +} + +/* Like makeFsmKeyArray except the result has only unique keys. They ordering + * will be changed. */ +void makeFsmUniqueKeyArray( KeySet &result, char *data, int len, + bool caseInsensitive, ParseData *pd ) +{ + /* Use a transitions list for getting unique keys. */ + if ( keyOps->isSigned ) { + /* Copy from a char star type. */ + char *src = data; + for ( int si = 0; si < len; si++ ) { + Key key( src[si] ); + result.insert( key ); + if ( caseInsensitive ) { + if ( key.isLower() ) + result.insert( key.toUpper() ); + else if ( key.isUpper() ) + result.insert( key.toLower() ); + } + } + } + else { + /* Copy from an unsigned byte ptr type. */ + unsigned char *src = (unsigned char*) data; + for ( int si = 0; si < len; si++ ) { + Key key( src[si] ); + result.insert( key ); + if ( caseInsensitive ) { + if ( key.isLower() ) + result.insert( key.toUpper() ); + else if ( key.isUpper() ) + result.insert( key.toLower() ); + } + } + } +} + +FsmAp *dotFsm( ParseData *pd ) +{ + FsmAp *retFsm = new FsmAp(); + retFsm->rangeFsm( keyOps->minKey, keyOps->maxKey ); + return retFsm; +} + +FsmAp *dotStarFsm( ParseData *pd ) +{ + FsmAp *retFsm = new FsmAp(); + retFsm->rangeStarFsm( keyOps->minKey, keyOps->maxKey ); + return retFsm; +} + +/* Make a builtin type. Depends on the signed nature of the alphabet type. */ +FsmAp *makeBuiltin( BuiltinMachine builtin, ParseData *pd ) +{ + /* FsmAp created to return. */ + FsmAp *retFsm = 0; + bool isSigned = keyOps->isSigned; + + switch ( builtin ) { + case BT_Any: { + /* All characters. */ + retFsm = dotFsm( pd ); + break; + } + case BT_Ascii: { + /* Ascii characters 0 to 127. */ + retFsm = new FsmAp(); + retFsm->rangeFsm( 0, 127 ); + break; + } + case BT_Extend: { + /* Ascii extended characters. This is the full byte range. Dependent + * on signed, vs no signed. If the alphabet is one byte then just use + * dot fsm. */ + if ( isSigned ) { + retFsm = new FsmAp(); + retFsm->rangeFsm( -128, 127 ); + } + else { + retFsm = new FsmAp(); + retFsm->rangeFsm( 0, 255 ); + } + break; + } + case BT_Alpha: { + /* Alpha [A-Za-z]. */ + FsmAp *upper = new FsmAp(), *lower = new FsmAp(); + upper->rangeFsm( 'A', 'Z' ); + lower->rangeFsm( 'a', 'z' ); + upper->unionOp( lower ); + upper->minimizePartition2(); + retFsm = upper; + break; + } + case BT_Digit: { + /* Digits [0-9]. */ + retFsm = new FsmAp(); + retFsm->rangeFsm( '0', '9' ); + break; + } + case BT_Alnum: { + /* Alpha numerics [0-9A-Za-z]. */ + FsmAp *digit = new FsmAp(), *lower = new FsmAp(); + FsmAp *upper = new FsmAp(); + digit->rangeFsm( '0', '9' ); + upper->rangeFsm( 'A', 'Z' ); + lower->rangeFsm( 'a', 'z' ); + digit->unionOp( upper ); + digit->unionOp( lower ); + digit->minimizePartition2(); + retFsm = digit; + break; + } + case BT_Lower: { + /* Lower case characters. */ + retFsm = new FsmAp(); + retFsm->rangeFsm( 'a', 'z' ); + break; + } + case BT_Upper: { + /* Upper case characters. */ + retFsm = new FsmAp(); + retFsm->rangeFsm( 'A', 'Z' ); + break; + } + case BT_Cntrl: { + /* Control characters. */ + FsmAp *cntrl = new FsmAp(); + FsmAp *highChar = new FsmAp(); + cntrl->rangeFsm( 0, 31 ); + highChar->concatFsm( 127 ); + cntrl->unionOp( highChar ); + cntrl->minimizePartition2(); + retFsm = cntrl; + break; + } + case BT_Graph: { + /* Graphical ascii characters [!-~]. */ + retFsm = new FsmAp(); + retFsm->rangeFsm( '!', '~' ); + break; + } + case BT_Print: { + /* Printable characters. Same as graph except includes space. */ + retFsm = new FsmAp(); + retFsm->rangeFsm( ' ', '~' ); + break; + } + case BT_Punct: { + /* Punctuation. */ + FsmAp *range1 = new FsmAp(); + FsmAp *range2 = new FsmAp(); + FsmAp *range3 = new FsmAp(); + FsmAp *range4 = new FsmAp(); + range1->rangeFsm( '!', '/' ); + range2->rangeFsm( ':', '@' ); + range3->rangeFsm( '[', '`' ); + range4->rangeFsm( '{', '~' ); + range1->unionOp( range2 ); + range1->unionOp( range3 ); + range1->unionOp( range4 ); + range1->minimizePartition2(); + retFsm = range1; + break; + } + case BT_Space: { + /* Whitespace: [\t\v\f\n\r ]. */ + FsmAp *cntrl = new FsmAp(); + FsmAp *space = new FsmAp(); + cntrl->rangeFsm( '\t', '\r' ); + space->concatFsm( ' ' ); + cntrl->unionOp( space ); + cntrl->minimizePartition2(); + retFsm = cntrl; + break; + } + case BT_Xdigit: { + /* Hex digits [0-9A-Fa-f]. */ + FsmAp *digit = new FsmAp(); + FsmAp *upper = new FsmAp(); + FsmAp *lower = new FsmAp(); + digit->rangeFsm( '0', '9' ); + upper->rangeFsm( 'A', 'F' ); + lower->rangeFsm( 'a', 'f' ); + digit->unionOp( upper ); + digit->unionOp( lower ); + digit->minimizePartition2(); + retFsm = digit; + break; + } + case BT_Lambda: { + retFsm = new FsmAp(); + retFsm->lambdaFsm(); + break; + } + case BT_Empty: { + retFsm = new FsmAp(); + retFsm->emptyFsm(); + break; + }} + + return retFsm; +} + +/* Check if this name inst or any name inst below is referenced. */ +bool NameInst::anyRefsRec() +{ + if ( numRefs > 0 ) + return true; + + /* Recurse on children until true. */ + for ( NameVect::Iter ch = childVect; ch.lte(); ch++ ) { + if ( (*ch)->anyRefsRec() ) + return true; + } + + return false; +} + +/* + * ParseData + */ + +/* Initialize the structure that will collect info during the parse of a + * machine. */ +ParseData::ParseData( const char *fileName, char *sectionName, + const InputLoc §ionLoc ) +: + sectionGraph(0), + generatingSectionSubset(false), + nextPriorKey(0), + /* 0 is reserved for global error actions. */ + nextLocalErrKey(1), + nextNameId(0), + nextCondId(0), + alphTypeSet(false), + getKeyExpr(0), + accessExpr(0), + prePushExpr(0), + postPopExpr(0), + pExpr(0), + peExpr(0), + eofExpr(0), + csExpr(0), + topExpr(0), + stackExpr(0), + actExpr(0), + tokstartExpr(0), + tokendExpr(0), + dataExpr(0), + lowerNum(0), + upperNum(0), + fileName(fileName), + sectionName(sectionName), + sectionLoc(sectionLoc), + curActionOrd(0), + curPriorOrd(0), + rootName(0), + exportsRootName(0), + nextEpsilonResolvedLink(0), + nextLongestMatchId(1), + lmRequiresErrorState(false), + cgd(0) +{ + /* Initialize the dictionary of graphs. This is our symbol table. The + * initialization needs to be done on construction which happens at the + * beginning of a machine spec so any assignment operators can reference + * the builtins. */ + initGraphDict(); +} + +/* Clean up the data collected during a parse. */ +ParseData::~ParseData() +{ + /* Delete all the nodes in the action list. Will cause all the + * string data that represents the actions to be deallocated. */ + actionList.empty(); +} + +/* Make a name id in the current name instantiation scope if it is not + * already there. */ +NameInst *ParseData::addNameInst( const InputLoc &loc, const char *data, bool isLabel ) +{ + /* Create the name instantitaion object and insert it. */ + NameInst *newNameInst = new NameInst( loc, curNameInst, data, nextNameId++, isLabel ); + curNameInst->childVect.append( newNameInst ); + if ( data != 0 ) + curNameInst->children.insertMulti( data, newNameInst ); + return newNameInst; +} + +void ParseData::initNameWalk() +{ + curNameInst = rootName; + curNameChild = 0; +} + +void ParseData::initExportsNameWalk() +{ + curNameInst = exportsRootName; + curNameChild = 0; +} + +/* Goes into the next child scope. The number of the child is already set up. + * We need this for the syncronous name tree and parse tree walk to work + * properly. It is reset on entry into a scope and advanced on poping of a + * scope. A call to enterNameScope should be accompanied by a corresponding + * popNameScope. */ +NameFrame ParseData::enterNameScope( bool isLocal, int numScopes ) +{ + /* Save off the current data. */ + NameFrame retFrame; + retFrame.prevNameInst = curNameInst; + retFrame.prevNameChild = curNameChild; + retFrame.prevLocalScope = localNameScope; + + /* Enter into the new name scope. */ + for ( int i = 0; i < numScopes; i++ ) { + curNameInst = curNameInst->childVect[curNameChild]; + curNameChild = 0; + } + + if ( isLocal ) + localNameScope = curNameInst; + + return retFrame; +} + +/* Return from a child scope to a parent. The parent info must be specified as + * an argument and is obtained from the corresponding call to enterNameScope. + * */ +void ParseData::popNameScope( const NameFrame &frame ) +{ + /* Pop the name scope. */ + curNameInst = frame.prevNameInst; + curNameChild = frame.prevNameChild+1; + localNameScope = frame.prevLocalScope; +} + +void ParseData::resetNameScope( const NameFrame &frame ) +{ + /* Pop the name scope. */ + curNameInst = frame.prevNameInst; + curNameChild = frame.prevNameChild; + localNameScope = frame.prevLocalScope; +} + + +void ParseData::unsetObsoleteEntries( FsmAp *graph ) +{ + /* Loop the reference names and increment the usage. Names that are no + * longer needed will be unset in graph. */ + for ( NameVect::Iter ref = curNameInst->referencedNames; ref.lte(); ref++ ) { + /* Get the name. */ + NameInst *name = *ref; + name->numUses += 1; + + /* If the name is no longer needed unset its corresponding entry. */ + if ( name->numUses == name->numRefs ) { + assert( graph->entryPoints.find( name->id ) != 0 ); + graph->unsetEntry( name->id ); + assert( graph->entryPoints.find( name->id ) == 0 ); + } + } +} + +NameSet ParseData::resolvePart( NameInst *refFrom, const char *data, bool recLabelsOnly ) +{ + /* Queue needed for breadth-first search, load it with the start node. */ + NameInstList nameQueue; + nameQueue.append( refFrom ); + + NameSet result; + while ( nameQueue.length() > 0 ) { + /* Pull the next from location off the queue. */ + NameInst *from = nameQueue.detachFirst(); + + /* Look for the name. */ + NameMapEl *low, *high; + if ( from->children.findMulti( data, low, high ) ) { + /* Record all instances of the name. */ + for ( ; low <= high; low++ ) + result.insert( low->value ); + } + + /* Name not there, do breadth-first operation of appending all + * childrent to the processing queue. */ + for ( NameVect::Iter name = from->childVect; name.lte(); name++ ) { + if ( !recLabelsOnly || (*name)->isLabel ) + nameQueue.append( *name ); + } + } + + /* Queue exhausted and name never found. */ + return result; +} + +void ParseData::resolveFrom( NameSet &result, NameInst *refFrom, + const NameRef &nameRef, int namePos ) +{ + /* Look for the name in the owning scope of the factor with aug. */ + NameSet partResult = resolvePart( refFrom, nameRef[namePos], false ); + + /* If there are more parts to the name then continue on. */ + if ( ++namePos < nameRef.length() ) { + /* There are more components to the name, search using all the part + * results as the base. */ + for ( NameSet::Iter name = partResult; name.lte(); name++ ) + resolveFrom( result, *name, nameRef, namePos ); + } + else { + /* This is the last component, append the part results to the final + * results. */ + result.insert( partResult ); + } +} + +/* Write out a name reference. */ +ostream &operator<<( ostream &out, const NameRef &nameRef ) +{ + int pos = 0; + if ( nameRef[pos] == 0 ) { + out << "::"; + pos += 1; + } + out << nameRef[pos++]; + for ( ; pos < nameRef.length(); pos++ ) + out << "::" << nameRef[pos]; + return out; +} + +ostream &operator<<( ostream &out, const NameInst &nameInst ) +{ + /* Count the number fully qualified name parts. */ + int numParents = 0; + NameInst *curParent = nameInst.parent; + while ( curParent != 0 ) { + numParents += 1; + curParent = curParent->parent; + } + + /* Make an array and fill it in. */ + curParent = nameInst.parent; + NameInst **parents = new NameInst*[numParents]; + for ( int p = numParents-1; p >= 0; p-- ) { + parents[p] = curParent; + curParent = curParent->parent; + } + + /* Write the parents out, skip the root. */ + for ( int p = 1; p < numParents; p++ ) + out << "::" << ( parents[p]->name != 0 ? parents[p]->name : "<ANON>" ); + + /* Write the name and cleanup. */ + out << "::" << ( nameInst.name != 0 ? nameInst.name : "<ANON>" ); + delete[] parents; + return out; +} + +struct CmpNameInstLoc +{ + static int compare( const NameInst *ni1, const NameInst *ni2 ) + { + if ( ni1->loc.line < ni2->loc.line ) + return -1; + else if ( ni1->loc.line > ni2->loc.line ) + return 1; + else if ( ni1->loc.col < ni2->loc.col ) + return -1; + else if ( ni1->loc.col > ni2->loc.col ) + return 1; + return 0; + } +}; + +void errorStateLabels( const NameSet &resolved ) +{ + MergeSort<NameInst*, CmpNameInstLoc> mergeSort; + mergeSort.sort( resolved.data, resolved.length() ); + for ( NameSet::Iter res = resolved; res.lte(); res++ ) + error((*res)->loc) << " -> " << **res << endl; +} + + +NameInst *ParseData::resolveStateRef( const NameRef &nameRef, InputLoc &loc, Action *action ) +{ + NameInst *nameInst = 0; + + /* Do the local search if the name is not strictly a root level name + * search. */ + if ( nameRef[0] != 0 ) { + /* If the action is referenced, resolve all of them. */ + if ( action != 0 && action->actionRefs.length() > 0 ) { + /* Look for the name in all referencing scopes. */ + NameSet resolved; + for ( ActionRefs::Iter actRef = action->actionRefs; actRef.lte(); actRef++ ) + resolveFrom( resolved, *actRef, nameRef, 0 ); + + if ( resolved.length() > 0 ) { + /* Take the first one. */ + nameInst = resolved[0]; + if ( resolved.length() > 1 ) { + /* Complain about the multiple references. */ + error(loc) << "state reference " << nameRef << + " resolves to multiple entry points" << endl; + errorStateLabels( resolved ); + } + } + } + } + + /* If not found in the local scope, look in global. */ + if ( nameInst == 0 ) { + NameSet resolved; + int fromPos = nameRef[0] != 0 ? 0 : 1; + resolveFrom( resolved, rootName, nameRef, fromPos ); + + if ( resolved.length() > 0 ) { + /* Take the first. */ + nameInst = resolved[0]; + if ( resolved.length() > 1 ) { + /* Complain about the multiple references. */ + error(loc) << "state reference " << nameRef << + " resolves to multiple entry points" << endl; + errorStateLabels( resolved ); + } + } + } + + if ( nameInst == 0 ) { + /* If not found then complain. */ + error(loc) << "could not resolve state reference " << nameRef << endl; + } + return nameInst; +} + +void ParseData::resolveNameRefs( InlineList *inlineList, Action *action ) +{ + for ( InlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case InlineItem::Entry: case InlineItem::Goto: + case InlineItem::Call: case InlineItem::Next: { + /* Resolve, pass action for local search. */ + NameInst *target = resolveStateRef( *item->nameRef, item->loc, action ); + + /* Name lookup error reporting is handled by resolveStateRef. */ + if ( target != 0 ) { + /* Check if the target goes into a longest match. */ + NameInst *search = target->parent; + while ( search != 0 ) { + if ( search->isLongestMatch ) { + error(item->loc) << "cannot enter inside a longest " + "match construction as an entry point" << endl; + break; + } + search = search->parent; + } + + /* Record the reference in the name. This will cause the + * entry point to survive to the end of the graph + * generating walk. */ + target->numRefs += 1; + } + + item->nameTarg = target; + break; + } + default: + break; + } + + /* Some of the item types may have children. */ + if ( item->children != 0 ) + resolveNameRefs( item->children, action ); + } +} + +/* Resolve references to labels in actions. */ +void ParseData::resolveActionNameRefs() +{ + for ( ActionList::Iter act = actionList; act.lte(); act++ ) { + /* Only care about the actions that are referenced. */ + if ( act->actionRefs.length() > 0 ) + resolveNameRefs( act->inlineList, act ); + } +} + +/* Walk a name tree starting at from and fill the name index. */ +void ParseData::fillNameIndex( NameInst *from ) +{ + /* Fill the value for from in the name index. */ + nameIndex[from->id] = from; + + /* Recurse on the implicit final state and then all children. */ + if ( from->final != 0 ) + fillNameIndex( from->final ); + for ( NameVect::Iter name = from->childVect; name.lte(); name++ ) + fillNameIndex( *name ); +} + +void ParseData::makeRootNames() +{ + /* Create the root name. */ + rootName = new NameInst( InputLoc(), 0, 0, nextNameId++, false ); + exportsRootName = new NameInst( InputLoc(), 0, 0, nextNameId++, false ); +} + +/* Build the name tree and supporting data structures. */ +void ParseData::makeNameTree( GraphDictEl *dictEl ) +{ + /* Set up curNameInst for the walk. */ + initNameWalk(); + + if ( dictEl != 0 ) { + /* A start location has been specified. */ + dictEl->value->makeNameTree( dictEl->loc, this ); + } + else { + /* First make the name tree. */ + for ( GraphList::Iter glel = instanceList; glel.lte(); glel++ ) { + /* Recurse on the instance. */ + glel->value->makeNameTree( glel->loc, this ); + } + } + + /* The number of nodes in the tree can now be given by nextNameId */ + nameIndex = new NameInst*[nextNameId]; + memset( nameIndex, 0, sizeof(NameInst*)*nextNameId ); + fillNameIndex( rootName ); + fillNameIndex( exportsRootName ); +} + + +void ParseData::createBuiltin( const char *name, BuiltinMachine builtin ) +{ + Expression *expression = new Expression( builtin ); + Join *join = new Join( expression ); + MachineDef *machineDef = new MachineDef( join ); + VarDef *varDef = new VarDef( name, machineDef ); + GraphDictEl *graphDictEl = new GraphDictEl( name, varDef ); + graphDict.insert( graphDictEl ); +} + +/* Initialize the graph dict with builtin types. */ +void ParseData::initGraphDict( ) +{ + createBuiltin( "any", BT_Any ); + createBuiltin( "ascii", BT_Ascii ); + createBuiltin( "extend", BT_Extend ); + createBuiltin( "alpha", BT_Alpha ); + createBuiltin( "digit", BT_Digit ); + createBuiltin( "alnum", BT_Alnum ); + createBuiltin( "lower", BT_Lower ); + createBuiltin( "upper", BT_Upper ); + createBuiltin( "cntrl", BT_Cntrl ); + createBuiltin( "graph", BT_Graph ); + createBuiltin( "print", BT_Print ); + createBuiltin( "punct", BT_Punct ); + createBuiltin( "space", BT_Space ); + createBuiltin( "xdigit", BT_Xdigit ); + createBuiltin( "null", BT_Lambda ); + createBuiltin( "zlen", BT_Lambda ); + createBuiltin( "empty", BT_Empty ); +} + +/* Set the alphabet type. If the types are not valid returns false. */ +bool ParseData::setAlphType( const InputLoc &loc, char *s1, char *s2 ) +{ + alphTypeLoc = loc; + userAlphType = findAlphType( s1, s2 ); + alphTypeSet = true; + return userAlphType != 0; +} + +/* Set the alphabet type. If the types are not valid returns false. */ +bool ParseData::setAlphType( const InputLoc &loc, char *s1 ) +{ + alphTypeLoc = loc; + userAlphType = findAlphType( s1 ); + alphTypeSet = true; + return userAlphType != 0; +} + +bool ParseData::setVariable( char *var, InlineList *inlineList ) +{ + bool set = true; + + if ( strcmp( var, "p" ) == 0 ) + pExpr = inlineList; + else if ( strcmp( var, "pe" ) == 0 ) + peExpr = inlineList; + else if ( strcmp( var, "eof" ) == 0 ) + eofExpr = inlineList; + else if ( strcmp( var, "cs" ) == 0 ) + csExpr = inlineList; + else if ( strcmp( var, "data" ) == 0 ) + dataExpr = inlineList; + else if ( strcmp( var, "top" ) == 0 ) + topExpr = inlineList; + else if ( strcmp( var, "stack" ) == 0 ) + stackExpr = inlineList; + else if ( strcmp( var, "act" ) == 0 ) + actExpr = inlineList; + else if ( strcmp( var, "ts" ) == 0 ) + tokstartExpr = inlineList; + else if ( strcmp( var, "te" ) == 0 ) + tokendExpr = inlineList; + else + set = false; + + return set; +} + +/* Initialize the key operators object that will be referenced by all fsms + * created. */ +void ParseData::initKeyOps( ) +{ + /* Signedness and bounds. */ + HostType *alphType = alphTypeSet ? userAlphType : hostLang->defaultAlphType; + thisKeyOps.setAlphType( alphType ); + + if ( lowerNum != 0 ) { + /* If ranges are given then interpret the alphabet type. */ + thisKeyOps.minKey = makeFsmKeyNum( lowerNum, rangeLowLoc, this ); + thisKeyOps.maxKey = makeFsmKeyNum( upperNum, rangeHighLoc, this ); + } + + thisCondData.lastCondKey = thisKeyOps.maxKey; +} + +void ParseData::printNameInst( NameInst *nameInst, int level ) +{ + for ( int i = 0; i < level; i++ ) + cerr << " "; + cerr << (nameInst->name != 0 ? nameInst->name : "<ANON>") << + " id: " << nameInst->id << + " refs: " << nameInst->numRefs << + " uses: " << nameInst->numUses << endl; + for ( NameVect::Iter name = nameInst->childVect; name.lte(); name++ ) + printNameInst( *name, level+1 ); +} + +/* Remove duplicates of unique actions from an action table. */ +void ParseData::removeDups( ActionTable &table ) +{ + /* Scan through the table looking for unique actions to + * remove duplicates of. */ + for ( int i = 0; i < table.length(); i++ ) { + /* Remove any duplicates ahead of i. */ + for ( int r = i+1; r < table.length(); ) { + if ( table[r].value == table[i].value ) + table.vremove(r); + else + r += 1; + } + } +} + +/* Remove duplicates from action lists. This operates only on transition and + * eof action lists and so should be called once all actions have been + * transfered to their final resting place. */ +void ParseData::removeActionDups( FsmAp *graph ) +{ + /* Loop all states. */ + for ( StateList::Iter state = graph->stateList; state.lte(); state++ ) { + /* Loop all transitions. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) + removeDups( trans->actionTable ); + removeDups( state->toStateActionTable ); + removeDups( state->fromStateActionTable ); + removeDups( state->eofActionTable ); + } +} + +Action *ParseData::newAction( const char *name, InlineList *inlineList ) +{ + InputLoc loc; + loc.line = 1; + loc.col = 1; + loc.fileName = "NONE"; + + Action *action = new Action( loc, name, inlineList, nextCondId++ ); + action->actionRefs.append( rootName ); + actionList.append( action ); + return action; +} + +void ParseData::initLongestMatchData() +{ + if ( lmList.length() > 0 ) { + /* The initTokStart action resets the token start. */ + InlineList *il1 = new InlineList; + il1->append( new InlineItem( InputLoc(), InlineItem::LmInitTokStart ) ); + initTokStart = newAction( "initts", il1 ); + initTokStart->isLmAction = true; + + /* The initActId action gives act a default value. */ + InlineList *il4 = new InlineList; + il4->append( new InlineItem( InputLoc(), InlineItem::LmInitAct ) ); + initActId = newAction( "initact", il4 ); + initActId->isLmAction = true; + + /* The setTokStart action sets tokstart. */ + InlineList *il5 = new InlineList; + il5->append( new InlineItem( InputLoc(), InlineItem::LmSetTokStart ) ); + setTokStart = newAction( "ts", il5 ); + setTokStart->isLmAction = true; + + /* The setTokEnd action sets tokend. */ + InlineList *il3 = new InlineList; + il3->append( new InlineItem( InputLoc(), InlineItem::LmSetTokEnd ) ); + setTokEnd = newAction( "te", il3 ); + setTokEnd->isLmAction = true; + + /* The action will also need an ordering: ahead of all user action + * embeddings. */ + initTokStartOrd = curActionOrd++; + initActIdOrd = curActionOrd++; + setTokStartOrd = curActionOrd++; + setTokEndOrd = curActionOrd++; + } +} + +/* After building the graph, do some extra processing to ensure the runtime + * data of the longest mactch operators is consistent. */ +void ParseData::setLongestMatchData( FsmAp *graph ) +{ + if ( lmList.length() > 0 ) { + /* Make sure all entry points (targets of fgoto, fcall, fnext, fentry) + * init the tokstart. */ + for ( EntryMap::Iter en = graph->entryPoints; en.lte(); en++ ) { + /* This is run after duplicates are removed, we must guard against + * inserting a duplicate. */ + ActionTable &actionTable = en->value->toStateActionTable; + if ( ! actionTable.hasAction( initTokStart ) ) + actionTable.setAction( initTokStartOrd, initTokStart ); + } + + /* Find the set of states that are the target of transitions with + * actions that have calls. These states will be targeted by fret + * statements. */ + StateSet states; + for ( StateList::Iter state = graph->stateList; state.lte(); state++ ) { + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + for ( ActionTable::Iter ati = trans->actionTable; ati.lte(); ati++ ) { + if ( ati->value->anyCall && trans->toState != 0 ) + states.insert( trans->toState ); + } + } + } + + + /* Init tokstart upon entering the above collected states. */ + for ( StateSet::Iter ps = states; ps.lte(); ps++ ) { + /* This is run after duplicates are removed, we must guard against + * inserting a duplicate. */ + ActionTable &actionTable = (*ps)->toStateActionTable; + if ( ! actionTable.hasAction( initTokStart ) ) + actionTable.setAction( initTokStartOrd, initTokStart ); + } + } +} + +/* Make the graph from a graph dict node. Does minimization and state sorting. */ +FsmAp *ParseData::makeInstance( GraphDictEl *gdNode ) +{ + /* Build the graph from a walk of the parse tree. */ + FsmAp *graph = gdNode->value->walk( this ); + + /* Resolve any labels that point to multiple states. Any labels that are + * still around are referenced only by gotos and calls and they need to be + * made into deterministic entry points. */ + graph->deterministicEntry(); + + /* + * All state construction is now complete. + */ + + /* Transfer actions from the out action tables to eof action tables. */ + for ( StateSet::Iter state = graph->finStateSet; state.lte(); state++ ) + graph->transferOutActions( *state ); + + /* Transfer global error actions. */ + for ( StateList::Iter state = graph->stateList; state.lte(); state++ ) + graph->transferErrorActions( state, 0 ); + + if ( ::wantDupsRemoved ) + removeActionDups( graph ); + + /* Remove unreachable states. There should be no dead end states. The + * subtract and intersection operators are the only places where they may + * be created and those operators clean them up. */ + graph->removeUnreachableStates(); + + /* No more fsm operations are to be done. Action ordering numbers are + * no longer of use and will just hinder minimization. Clear them. */ + graph->nullActionKeys(); + + /* Transition priorities are no longer of use. We can clear them + * because they will just hinder minimization as well. Clear them. */ + graph->clearAllPriorities(); + + if ( minimizeOpt != MinimizeNone ) { + /* Minimize here even if we minimized at every op. Now that function + * keys have been cleared we may get a more minimal fsm. */ + switch ( minimizeLevel ) { + case MinimizeApprox: + graph->minimizeApproximate(); + break; + case MinimizeStable: + graph->minimizeStable(); + break; + case MinimizePartition1: + graph->minimizePartition1(); + break; + case MinimizePartition2: + graph->minimizePartition2(); + break; + } + } + + graph->compressTransitions(); + + return graph; +} + +void ParseData::printNameTree() +{ + /* Print the name instance map. */ + for ( NameVect::Iter name = rootName->childVect; name.lte(); name++ ) + printNameInst( *name, 0 ); + + cerr << "name index:" << endl; + /* Show that the name index is correct. */ + for ( int ni = 0; ni < nextNameId; ni++ ) { + cerr << ni << ": "; + const char *name = nameIndex[ni]->name; + cerr << ( name != 0 ? name : "<ANON>" ) << endl; + } +} + +FsmAp *ParseData::makeSpecific( GraphDictEl *gdNode ) +{ + /* Build the name tree and supporting data structures. */ + makeNameTree( gdNode ); + + /* Resove name references from gdNode. */ + initNameWalk(); + gdNode->value->resolveNameRefs( this ); + + /* Do not resolve action references. Since we are not building the entire + * graph there's a good chance that many name references will fail. This + * is okay since generating part of the graph is usually only done when + * inspecting the compiled machine. */ + + /* Same story for extern entry point references. */ + + /* Flag this case so that the XML code generator is aware that we haven't + * looked up name references in actions. It can then avoid segfaulting. */ + generatingSectionSubset = true; + + /* Just building the specified graph. */ + initNameWalk(); + FsmAp *mainGraph = makeInstance( gdNode ); + + return mainGraph; +} + +FsmAp *ParseData::makeAll() +{ + /* Build the name tree and supporting data structures. */ + makeNameTree( 0 ); + + /* Resove name references in the tree. */ + initNameWalk(); + for ( GraphList::Iter glel = instanceList; glel.lte(); glel++ ) + glel->value->resolveNameRefs( this ); + + /* Resolve action code name references. */ + resolveActionNameRefs(); + + /* Force name references to the top level instantiations. */ + for ( NameVect::Iter inst = rootName->childVect; inst.lte(); inst++ ) + (*inst)->numRefs += 1; + + FsmAp *mainGraph = 0; + FsmAp **graphs = new FsmAp*[instanceList.length()]; + int numOthers = 0; + + /* Make all the instantiations, we know that main exists in this list. */ + initNameWalk(); + for ( GraphList::Iter glel = instanceList; glel.lte(); glel++ ) { + if ( strcmp( glel->key, mainMachine ) == 0 ) { + /* Main graph is always instantiated. */ + mainGraph = makeInstance( glel ); + } + else { + /* Instantiate and store in others array. */ + graphs[numOthers++] = makeInstance( glel ); + } + } + + if ( mainGraph == 0 ) + mainGraph = graphs[--numOthers]; + + if ( numOthers > 0 ) { + /* Add all the other graphs into main. */ + mainGraph->globOp( graphs, numOthers ); + } + + delete[] graphs; + return mainGraph; +} + +void ParseData::analyzeAction( Action *action, InlineList *inlineList ) +{ + /* FIXME: Actions used as conditions should be very constrained. */ + for ( InlineList::Iter item = *inlineList; item.lte(); item++ ) { + if ( item->type == InlineItem::Call || item->type == InlineItem::CallExpr ) + action->anyCall = true; + + /* Need to recurse into longest match items. */ + if ( item->type == InlineItem::LmSwitch ) { + LongestMatch *lm = item->longestMatch; + for ( LmPartList::Iter lmi = *lm->longestMatchList; lmi.lte(); lmi++ ) { + if ( lmi->action != 0 ) + analyzeAction( action, lmi->action->inlineList ); + } + } + + if ( item->type == InlineItem::LmOnLast || + item->type == InlineItem::LmOnNext || + item->type == InlineItem::LmOnLagBehind ) + { + LongestMatchPart *lmi = item->longestMatchPart; + if ( lmi->action != 0 ) + analyzeAction( action, lmi->action->inlineList ); + } + + if ( item->children != 0 ) + analyzeAction( action, item->children ); + } +} + + +/* Check actions for bad uses of fsm directives. We don't go inside longest + * match items in actions created by ragel, since we just want the user + * actions. */ +void ParseData::checkInlineList( Action *act, InlineList *inlineList ) +{ + for ( InlineList::Iter item = *inlineList; item.lte(); item++ ) { + /* EOF checks. */ + if ( act->numEofRefs > 0 ) { + switch ( item->type ) { + /* Currently no checks. */ + default: + break; + } + } + + /* Recurse. */ + if ( item->children != 0 ) + checkInlineList( act, item->children ); + } +} + +void ParseData::checkAction( Action *action ) +{ + /* Check for actions with calls that are embedded within a longest match + * machine. */ + if ( !action->isLmAction && action->numRefs() > 0 && action->anyCall ) { + for ( ActionRefs::Iter ar = action->actionRefs; ar.lte(); ar++ ) { + NameInst *check = *ar; + while ( check != 0 ) { + if ( check->isLongestMatch ) { + error(action->loc) << "within a scanner, fcall is permitted" + " only in pattern actions" << endl; + break; + } + check = check->parent; + } + } + } + + checkInlineList( action, action->inlineList ); +} + + +void ParseData::analyzeGraph( FsmAp *graph ) +{ + for ( ActionList::Iter act = actionList; act.lte(); act++ ) + analyzeAction( act, act->inlineList ); + + for ( StateList::Iter st = graph->stateList; st.lte(); st++ ) { + /* The transition list. */ + for ( TransList::Iter trans = st->outList; trans.lte(); trans++ ) { + for ( ActionTable::Iter at = trans->actionTable; at.lte(); at++ ) + at->value->numTransRefs += 1; + } + + for ( ActionTable::Iter at = st->toStateActionTable; at.lte(); at++ ) + at->value->numToStateRefs += 1; + + for ( ActionTable::Iter at = st->fromStateActionTable; at.lte(); at++ ) + at->value->numFromStateRefs += 1; + + for ( ActionTable::Iter at = st->eofActionTable; at.lte(); at++ ) + at->value->numEofRefs += 1; + + for ( StateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + for ( CondSet::Iter sci = sc->condSpace->condSet; sci.lte(); sci++ ) + (*sci)->numCondRefs += 1; + } + } + + /* Checks for bad usage of directives in action code. */ + for ( ActionList::Iter act = actionList; act.lte(); act++ ) + checkAction( act ); +} + +void ParseData::makeExportsNameTree() +{ + /* Make a name tree for the exports. */ + initExportsNameWalk(); + + /* First make the name tree. */ + for ( GraphDict::Iter gdel = graphDict; gdel.lte(); gdel++ ) { + if ( gdel->value->isExport ) { + /* Recurse on the instance. */ + gdel->value->makeNameTree( gdel->loc, this ); + } + } +} + +void ParseData::makeExports() +{ + makeExportsNameTree(); + + /* Resove name references in the tree. */ + initExportsNameWalk(); + for ( GraphDict::Iter gdel = graphDict; gdel.lte(); gdel++ ) { + if ( gdel->value->isExport ) + gdel->value->resolveNameRefs( this ); + } + + /* Make all the instantiations, we know that main exists in this list. */ + initExportsNameWalk(); + for ( GraphDict::Iter gdel = graphDict; gdel.lte(); gdel++ ) { + /* Check if this var def is an export. */ + if ( gdel->value->isExport ) { + /* Build the graph from a walk of the parse tree. */ + FsmAp *graph = gdel->value->walk( this ); + + /* Build the graph from a walk of the parse tree. */ + if ( !graph->checkSingleCharMachine() ) { + error(gdel->loc) << "bad export machine, must define " + "a single character" << endl; + } + else { + /* Safe to extract the key and declare the export. */ + Key exportKey = graph->startState->outList.head->lowKey; + exportList.append( new Export( gdel->value->name, exportKey ) ); + } + } + } + +} + +/* Construct the machine and catch failures which can occur during + * construction. */ +void ParseData::prepareMachineGen( GraphDictEl *graphDictEl ) +{ + try { + /* This machine construction can fail. */ + prepareMachineGenTBWrapped( graphDictEl ); + } + catch ( FsmConstructFail fail ) { + switch ( fail.reason ) { + case FsmConstructFail::CondNoKeySpace: { + InputLoc &loc = alphTypeSet ? alphTypeLoc : sectionLoc; + error(loc) << "sorry, no more characters are " + "available in the alphabet space" << endl; + error(loc) << " for conditions, please use a " + "smaller alphtype or reduce" << endl; + error(loc) << " the span of characters on which " + "conditions are embedded" << endl; + break; + } + } + } +} + +void ParseData::prepareMachineGenTBWrapped( GraphDictEl *graphDictEl ) +{ + beginProcessing(); + initKeyOps(); + makeRootNames(); + initLongestMatchData(); + + /* Make the graph, do minimization. */ + if ( graphDictEl == 0 ) + sectionGraph = makeAll(); + else + sectionGraph = makeSpecific( graphDictEl ); + + /* Compute exports from the export definitions. */ + makeExports(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + analyzeGraph( sectionGraph ); + + /* Depends on the graph analysis. */ + setLongestMatchData( sectionGraph ); + + /* Decide if an error state is necessary. + * 1. There is an error transition + * 2. There is a gap in the transitions + * 3. The longest match operator requires it. */ + if ( lmRequiresErrorState || sectionGraph->hasErrorTrans() ) + sectionGraph->errState = sectionGraph->addState(); + + /* State numbers need to be assigned such that all final states have a + * larger state id number than all non-final states. This enables the + * first_final mechanism to function correctly. We also want states to be + * ordered in a predictable fashion. So we first apply a depth-first + * search, then do a stable sort by final state status, then assign + * numbers. */ + + sectionGraph->depthFirstOrdering(); + sectionGraph->sortStatesByFinal(); + sectionGraph->setStateNumbers( 0 ); +} + +void ParseData::generateReduced( InputData &inputData ) +{ + beginProcessing(); + + cgd = makeCodeGen( inputData.inputFileName, sectionName, *inputData.outStream ); + + /* Make the generator. */ + BackendGen backendGen( sectionName, this, sectionGraph, cgd ); + + /* Write out with it. */ + backendGen.makeBackend(); + + if ( printStatistics ) { + cerr << "fsm name : " << sectionName << endl; + cerr << "num states: " << sectionGraph->stateList.length() << endl; + cerr << endl; + } +} + +void ParseData::generateXML( ostream &out ) +{ + beginProcessing(); + + /* Make the generator. */ + XMLCodeGen codeGen( sectionName, this, sectionGraph, out ); + + /* Write out with it. */ + codeGen.writeXML(); + + if ( printStatistics ) { + cerr << "fsm name : " << sectionName << endl; + cerr << "num states: " << sectionGraph->stateList.length() << endl; + cerr << endl; + } +} + diff --git a/contrib/tools/ragel6/parsedata.h b/contrib/tools/ragel6/parsedata.h new file mode 100644 index 0000000000..3e3bfa6e01 --- /dev/null +++ b/contrib/tools/ragel6/parsedata.h @@ -0,0 +1,391 @@ +/* + * Copyright 2001-2008 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _PARSEDATA_H +#define _PARSEDATA_H + +#include <iostream> +#include <limits.h> +#include <sstream> +#include "avlmap.h" +#include "bstmap.h" +#include "vector.h" +#include "dlist.h" +#include "fsmgraph.h" +#include "compare.h" +#include "vector.h" +#include "common.h" +#include "parsetree.h" + +/* Forwards. */ +using std::ostream; + +struct VarDef; +struct Join; +struct Expression; +struct Term; +struct FactorWithAug; +struct FactorWithLabel; +struct FactorWithRep; +struct FactorWithNeg; +struct Factor; +struct Literal; +struct Range; +struct RegExpr; +struct ReItem; +struct ReOrBlock; +struct ReOrItem; +struct LongestMatch; +struct InputData; +struct CodeGenData; +typedef DList<LongestMatch> LmList; + + +/* Graph dictionary. */ +struct GraphDictEl +: + public AvlTreeEl<GraphDictEl>, + public DListEl<GraphDictEl> +{ + GraphDictEl( const char *k ) + : key(k), value(0), isInstance(false) { } + GraphDictEl( const char *k, VarDef *value ) + : key(k), value(value), isInstance(false) { } + + const char *getKey() { return key; } + + const char *key; + VarDef *value; + bool isInstance; + + /* Location info of graph definition. Points to variable name of assignment. */ + InputLoc loc; +}; + +typedef AvlTree<GraphDictEl, const char*, CmpStr> GraphDict; +typedef DList<GraphDictEl> GraphList; + +/* Priority name dictionary. */ +typedef AvlMapEl<char*, int> PriorDictEl; +typedef AvlMap<char*, int, CmpStr> PriorDict; + +/* Local error name dictionary. */ +typedef AvlMapEl<const char*, int> LocalErrDictEl; +typedef AvlMap<const char*, int, CmpStr> LocalErrDict; + +/* Tree of instantiated names. */ +typedef BstMapEl<const char*, NameInst*> NameMapEl; +typedef BstMap<const char*, NameInst*, CmpStr> NameMap; +typedef Vector<NameInst*> NameVect; +typedef BstSet<NameInst*> NameSet; + +/* Node in the tree of instantiated names. */ +struct NameInst +{ + NameInst( const InputLoc &loc, NameInst *parent, const char *name, int id, bool isLabel ) : + loc(loc), parent(parent), name(name), id(id), isLabel(isLabel), + isLongestMatch(false), numRefs(0), numUses(0), start(0), final(0) {} + + InputLoc loc; + + /* Keep parent pointers in the name tree to retrieve + * fully qulified names. */ + NameInst *parent; + + const char *name; + int id; + bool isLabel; + bool isLongestMatch; + + int numRefs; + int numUses; + + /* Names underneath us, excludes anonymous names. */ + NameMap children; + + /* All names underneath us in order of appearance. */ + NameVect childVect; + + /* Join scopes need an implicit "final" target. */ + NameInst *start, *final; + + /* During a fsm generation walk, lists the names that are referenced by + * epsilon operations in the current scope. After the link is made by the + * epsilon reference and the join operation is complete, the label can + * have its refcount decremented. Once there are no more references the + * entry point can be removed from the fsm returned. */ + NameVect referencedNames; + + /* Pointers for the name search queue. */ + NameInst *prev, *next; + + /* Check if this name inst or any name inst below is referenced. */ + bool anyRefsRec(); +}; + +typedef DList<NameInst> NameInstList; + +/* Stack frame used in walking the name tree. */ +struct NameFrame +{ + NameInst *prevNameInst; + int prevNameChild; + NameInst *prevLocalScope; +}; + +struct LengthDef +{ + LengthDef( char *name ) + : name(name) {} + + char *name; + LengthDef *prev, *next; +}; + +typedef DList<LengthDef> LengthDefList; + +/* Class to collect information about the machine during the + * parse of input. */ +struct ParseData +{ + /* Create a new parse data object. This is done at the beginning of every + * fsm specification. */ + ParseData( const char *fileName, char *sectionName, const InputLoc §ionLoc ); + ~ParseData(); + + /* + * Setting up the graph dict. + */ + + /* Initialize a graph dict with the basic fsms. */ + void initGraphDict(); + void createBuiltin( const char *name, BuiltinMachine builtin ); + + /* Make a name id in the current name instantiation scope if it is not + * already there. */ + NameInst *addNameInst( const InputLoc &loc, const char *data, bool isLabel ); + void makeRootNames(); + void makeNameTree( GraphDictEl *gdNode ); + void makeExportsNameTree(); + void fillNameIndex( NameInst *from ); + void printNameTree(); + + /* Increments the usage count on entry names. Names that are no longer + * needed will have their entry points unset. */ + void unsetObsoleteEntries( FsmAp *graph ); + + /* Resove name references in action code and epsilon transitions. */ + NameSet resolvePart( NameInst *refFrom, const char *data, bool recLabelsOnly ); + void resolveFrom( NameSet &result, NameInst *refFrom, + const NameRef &nameRef, int namePos ); + NameInst *resolveStateRef( const NameRef &nameRef, InputLoc &loc, Action *action ); + void resolveNameRefs( InlineList *inlineList, Action *action ); + void resolveActionNameRefs(); + + /* Set the alphabet type. If type types are not valid returns false. */ + bool setAlphType( const InputLoc &loc, char *s1, char *s2 ); + bool setAlphType( const InputLoc &loc, char *s1 ); + + /* Override one of the variables ragel uses. */ + bool setVariable( char *var, InlineList *inlineList ); + + /* Unique actions. */ + void removeDups( ActionTable &actionTable ); + void removeActionDups( FsmAp *graph ); + + /* Dumping the name instantiation tree. */ + void printNameInst( NameInst *nameInst, int level ); + + /* Make the graph from a graph dict node. Does minimization. */ + FsmAp *makeInstance( GraphDictEl *gdNode ); + FsmAp *makeSpecific( GraphDictEl *gdNode ); + FsmAp *makeAll(); + + /* Checking the contents of actions. */ + void checkAction( Action *action ); + void checkInlineList( Action *act, InlineList *inlineList ); + + void analyzeAction( Action *action, InlineList *inlineList ); + void analyzeGraph( FsmAp *graph ); + void makeExports(); + + void prepareMachineGen( GraphDictEl *graphDictEl ); + void prepareMachineGenTBWrapped( GraphDictEl *graphDictEl ); + void generateXML( ostream &out ); + void generateReduced( InputData &inputData ); + FsmAp *sectionGraph; + bool generatingSectionSubset; + + void initKeyOps(); + + /* + * Data collected during the parse. + */ + + /* Dictionary of graphs. Both instances and non-instances go here. */ + GraphDict graphDict; + + /* The list of instances. */ + GraphList instanceList; + + /* Dictionary of actions. Lets actions be defined and then referenced. */ + ActionDict actionDict; + + /* Dictionary of named priorities. */ + PriorDict priorDict; + + /* Dictionary of named local errors. */ + LocalErrDict localErrDict; + + /* List of actions. Will be pasted into a switch statement. */ + ActionList actionList; + + /* The id of the next priority name and label. */ + int nextPriorKey, nextLocalErrKey, nextNameId, nextCondId; + + /* The default priority number key for a machine. This is active during + * the parse of the rhs of a machine assignment. */ + int curDefPriorKey; + + int curDefLocalErrKey; + + /* Alphabet type. */ + HostType *userAlphType; + bool alphTypeSet; + InputLoc alphTypeLoc; + + /* Element type and get key expression. */ + InlineList *getKeyExpr; + InlineList *accessExpr; + + /* Stack management */ + InlineList *prePushExpr; + InlineList *postPopExpr; + + /* Overriding variables. */ + InlineList *pExpr; + InlineList *peExpr; + InlineList *eofExpr; + InlineList *csExpr; + InlineList *topExpr; + InlineList *stackExpr; + InlineList *actExpr; + InlineList *tokstartExpr; + InlineList *tokendExpr; + InlineList *dataExpr; + + /* The alphabet range. */ + char *lowerNum, *upperNum; + Key lowKey, highKey; + InputLoc rangeLowLoc, rangeHighLoc; + + /* The name of the file the fsm is from, and the spec name. */ + const char *fileName; + char *sectionName; + InputLoc sectionLoc; + + /* Counting the action and priority ordering. */ + int curActionOrd; + int curPriorOrd; + + /* Root of the name tree. One root is for the instantiated machines. The + * other root is for exported definitions. */ + NameInst *rootName; + NameInst *exportsRootName; + + /* Name tree walking. */ + NameInst *curNameInst; + int curNameChild; + + /* The place where resolved epsilon transitions go. These cannot go into + * the parse tree because a single epsilon op can resolve more than once + * to different nameInsts if the machine it's in is used more than once. */ + NameVect epsilonResolvedLinks; + int nextEpsilonResolvedLink; + + /* Root of the name tree used for doing local name searches. */ + NameInst *localNameScope; + + void setLmInRetLoc( InlineList *inlineList ); + void initLongestMatchData(); + void setLongestMatchData( FsmAp *graph ); + void initNameWalk(); + void initExportsNameWalk(); + NameInst *nextNameScope() { return curNameInst->childVect[curNameChild]; } + NameFrame enterNameScope( bool isLocal, int numScopes ); + void popNameScope( const NameFrame &frame ); + void resetNameScope( const NameFrame &frame ); + + /* Make name ids to name inst pointers. */ + NameInst **nameIndex; + + /* Counter for assigning ids to longest match items. */ + int nextLongestMatchId; + bool lmRequiresErrorState; + + /* List of all longest match parse tree items. */ + LmList lmList; + + Action *newAction( const char *name, InlineList *inlineList ); + + Action *initTokStart; + int initTokStartOrd; + + Action *setTokStart; + int setTokStartOrd; + + Action *initActId; + int initActIdOrd; + + Action *setTokEnd; + int setTokEndOrd; + + void beginProcessing() + { + ::condData = &thisCondData; + ::keyOps = &thisKeyOps; + } + + CondData thisCondData; + KeyOps thisKeyOps; + + ExportList exportList; + LengthDefList lengthDefList; + + CodeGenData *cgd; +}; + +void afterOpMinimize( FsmAp *fsm, bool lastInSeq = true ); +Key makeFsmKeyHex( char *str, const InputLoc &loc, ParseData *pd ); +Key makeFsmKeyDec( char *str, const InputLoc &loc, ParseData *pd ); +Key makeFsmKeyNum( char *str, const InputLoc &loc, ParseData *pd ); +Key makeFsmKeyChar( char c, ParseData *pd ); +void makeFsmKeyArray( Key *result, char *data, int len, ParseData *pd ); +void makeFsmUniqueKeyArray( KeySet &result, char *data, int len, + bool caseInsensitive, ParseData *pd ); +FsmAp *makeBuiltin( BuiltinMachine builtin, ParseData *pd ); +FsmAp *dotFsm( ParseData *pd ); +FsmAp *dotStarFsm( ParseData *pd ); + +void errorStateLabels( const NameSet &locations ); + + +#endif diff --git a/contrib/tools/ragel6/parsetree.cpp b/contrib/tools/ragel6/parsetree.cpp new file mode 100644 index 0000000000..ff538aa28e --- /dev/null +++ b/contrib/tools/ragel6/parsetree.cpp @@ -0,0 +1,2139 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iostream> +#include <iomanip> +#include <errno.h> +#include <limits.h> +#include <stdlib.h> + +/* Parsing. */ +#include "ragel.h" +#include "rlparse.h" +#include "parsetree.h" + +using namespace std; +ostream &operator<<( ostream &out, const NameRef &nameRef ); +ostream &operator<<( ostream &out, const NameInst &nameInst ); + +/* Convert the literal string which comes in from the scanner into an array of + * characters with escapes and options interpreted. Also null terminates the + * string. Though this null termination should not be relied on for + * interpreting literals in the parser because the string may contain \0 */ +char *prepareLitString( const InputLoc &loc, const char *data, long length, + long &resLen, bool &caseInsensitive ) +{ + char *resData = new char[length+1]; + caseInsensitive = false; + + const char *src = data + 1; + const char *end = data + length - 1; + + while ( *end != '\'' && *end != '\"' ) { + if ( *end == 'i' ) + caseInsensitive = true; + else { + error( loc ) << "literal string '" << *end << + "' option not supported" << endl; + } + end -= 1; + } + + char *dest = resData; + long len = 0; + while ( src != end ) { + if ( *src == '\\' ) { + switch ( src[1] ) { + case '0': dest[len++] = '\0'; break; + case 'a': dest[len++] = '\a'; break; + case 'b': dest[len++] = '\b'; break; + case 't': dest[len++] = '\t'; break; + case 'n': dest[len++] = '\n'; break; + case 'v': dest[len++] = '\v'; break; + case 'f': dest[len++] = '\f'; break; + case 'r': dest[len++] = '\r'; break; + case '\n': break; + default: dest[len++] = src[1]; break; + } + src += 2; + } + else { + dest[len++] = *src++; + } + } + + resLen = len; + resData[resLen] = 0; + return resData; +} + +FsmAp *VarDef::walk( ParseData *pd ) +{ + /* We enter into a new name scope. */ + NameFrame nameFrame = pd->enterNameScope( true, 1 ); + + /* Recurse on the expression. */ + FsmAp *rtnVal = machineDef->walk( pd ); + + /* Do the tranfer of local error actions. */ + LocalErrDictEl *localErrDictEl = pd->localErrDict.find( name ); + if ( localErrDictEl != 0 ) { + for ( StateList::Iter state = rtnVal->stateList; state.lte(); state++ ) + rtnVal->transferErrorActions( state, localErrDictEl->value ); + } + + /* If the expression below is a join operation with multiple expressions + * then it just had epsilon transisions resolved. If it is a join + * with only a single expression then run the epsilon op now. */ + if ( machineDef->type == MachineDef::JoinType && machineDef->join->exprList.length() == 1 ) + rtnVal->epsilonOp(); + + /* We can now unset entry points that are not longer used. */ + pd->unsetObsoleteEntries( rtnVal ); + + /* If the name of the variable is referenced then add the entry point to + * the graph. */ + if ( pd->curNameInst->numRefs > 0 ) + rtnVal->setEntry( pd->curNameInst->id, rtnVal->startState ); + + /* Pop the name scope. */ + pd->popNameScope( nameFrame ); + return rtnVal; +} + +void VarDef::makeNameTree( const InputLoc &loc, ParseData *pd ) +{ + /* The variable definition enters a new scope. */ + NameInst *prevNameInst = pd->curNameInst; + pd->curNameInst = pd->addNameInst( loc, name, false ); + + if ( machineDef->type == MachineDef::LongestMatchType ) + pd->curNameInst->isLongestMatch = true; + + /* Recurse. */ + machineDef->makeNameTree( pd ); + + /* The name scope ends, pop the name instantiation. */ + pd->curNameInst = prevNameInst; +} + +void VarDef::resolveNameRefs( ParseData *pd ) +{ + /* Entering into a new scope. */ + NameFrame nameFrame = pd->enterNameScope( true, 1 ); + + /* Recurse. */ + machineDef->resolveNameRefs( pd ); + + /* The name scope ends, pop the name instantiation. */ + pd->popNameScope( nameFrame ); +} + +InputLoc LongestMatchPart::getLoc() +{ + return action != 0 ? action->loc : semiLoc; +} + +/* + * If there are any LMs then all of the following entry points must reset + * tokstart: + * + * 1. fentry(StateRef) + * 2. ftoto(StateRef), fcall(StateRef), fnext(StateRef) + * 3. targt of any transition that has an fcall (the return loc). + * 4. start state of all longest match routines. + */ + +Action *LongestMatch::newAction( ParseData *pd, const InputLoc &loc, + const char *name, InlineList *inlineList ) +{ + Action *action = new Action( loc, name, inlineList, pd->nextCondId++ ); + action->actionRefs.append( pd->curNameInst ); + pd->actionList.append( action ); + action->isLmAction = true; + return action; +} + +void LongestMatch::makeActions( ParseData *pd ) +{ + /* Make actions that set the action id. */ + for ( LmPartList::Iter lmi = *longestMatchList; lmi.lte(); lmi++ ) { + /* For each part create actions for setting the match type. We need + * to do this so that the actions will go into the actionIndex. */ + InlineList *inlineList = new InlineList; + inlineList->append( new InlineItem( lmi->getLoc(), this, lmi, + InlineItem::LmSetActId ) ); + char *actName = new char[50]; + sprintf( actName, "store%i", lmi->longestMatchId ); + lmi->setActId = newAction( pd, lmi->getLoc(), actName, inlineList ); + } + + /* Make actions that execute the user action and restart on the last + * character. */ + for ( LmPartList::Iter lmi = *longestMatchList; lmi.lte(); lmi++ ) { + /* For each part create actions for setting the match type. We need + * to do this so that the actions will go into the actionIndex. */ + InlineList *inlineList = new InlineList; + inlineList->append( new InlineItem( lmi->getLoc(), this, lmi, + InlineItem::LmOnLast ) ); + char *actName = new char[50]; + sprintf( actName, "last%i", lmi->longestMatchId ); + lmi->actOnLast = newAction( pd, lmi->getLoc(), actName, inlineList ); + } + + /* Make actions that execute the user action and restart on the next + * character. These actions will set tokend themselves (it is the current + * char). */ + for ( LmPartList::Iter lmi = *longestMatchList; lmi.lte(); lmi++ ) { + /* For each part create actions for setting the match type. We need + * to do this so that the actions will go into the actionIndex. */ + InlineList *inlineList = new InlineList; + inlineList->append( new InlineItem( lmi->getLoc(), this, lmi, + InlineItem::LmOnNext ) ); + char *actName = new char[50]; + sprintf( actName, "next%i", lmi->longestMatchId ); + lmi->actOnNext = newAction( pd, lmi->getLoc(), actName, inlineList ); + } + + /* Make actions that execute the user action and restart at tokend. These + * actions execute some time after matching the last char. */ + for ( LmPartList::Iter lmi = *longestMatchList; lmi.lte(); lmi++ ) { + /* For each part create actions for setting the match type. We need + * to do this so that the actions will go into the actionIndex. */ + InlineList *inlineList = new InlineList; + inlineList->append( new InlineItem( lmi->getLoc(), this, lmi, + InlineItem::LmOnLagBehind ) ); + char *actName = new char[50]; + sprintf( actName, "lag%i", lmi->longestMatchId ); + lmi->actLagBehind = newAction( pd, lmi->getLoc(), actName, inlineList ); + } + + InputLoc loc; + loc.line = 1; + loc.col = 1; + loc.fileName = "NONE"; + + /* Create the error action. */ + InlineList *il6 = new InlineList; + il6->append( new InlineItem( loc, this, 0, InlineItem::LmSwitch ) ); + lmActSelect = newAction( pd, loc, "switch", il6 ); +} + +void LongestMatch::findName( ParseData *pd ) +{ + NameInst *nameInst = pd->curNameInst; + while ( nameInst->name == 0 ) { + nameInst = nameInst->parent; + /* Since every machine must must have a name, we should always find a + * name for the longest match. */ + assert( nameInst != 0 ); + } + name = nameInst->name; +} + +void LongestMatch::makeNameTree( ParseData *pd ) +{ + /* Create an anonymous scope for the longest match. Will be used for + * restarting machine after matching a token. */ + NameInst *prevNameInst = pd->curNameInst; + pd->curNameInst = pd->addNameInst( loc, 0, false ); + + /* Recurse into all parts of the longest match operator. */ + for ( LmPartList::Iter lmi = *longestMatchList; lmi.lte(); lmi++ ) + lmi->join->makeNameTree( pd ); + + /* Traverse the name tree upwards to find a name for this lm. */ + findName( pd ); + + /* Also make the longest match's actions at this point. */ + makeActions( pd ); + + /* The name scope ends, pop the name instantiation. */ + pd->curNameInst = prevNameInst; +} + +void LongestMatch::resolveNameRefs( ParseData *pd ) +{ + /* The longest match gets its own name scope. */ + NameFrame nameFrame = pd->enterNameScope( true, 1 ); + + /* Take an action reference for each longest match item and recurse. */ + for ( LmPartList::Iter lmi = *longestMatchList; lmi.lte(); lmi++ ) { + /* Record the reference if the item has an action. */ + if ( lmi->action != 0 ) + lmi->action->actionRefs.append( pd->localNameScope ); + + /* Recurse down the join. */ + lmi->join->resolveNameRefs( pd ); + } + + /* The name scope ends, pop the name instantiation. */ + pd->popNameScope( nameFrame ); +} + +void LongestMatch::restart( FsmAp *graph, TransAp *trans ) +{ + StateAp *fromState = trans->fromState; + graph->detachTrans( fromState, trans->toState, trans ); + graph->attachTrans( fromState, graph->startState, trans ); +} + +void LongestMatch::runLongestMatch( ParseData *pd, FsmAp *graph ) +{ + graph->markReachableFromHereStopFinal( graph->startState ); + for ( StateList::Iter ms = graph->stateList; ms.lte(); ms++ ) { + if ( ms->stateBits & STB_ISMARKED ) { + ms->lmItemSet.insert( 0 ); + ms->stateBits &= ~ STB_ISMARKED; + } + } + + /* Transfer the first item of non-empty lmAction tables to the item sets + * of the states that follow. Exclude states that have no transitions out. + * This must happen on a separate pass so that on each iteration of the + * next pass we have the item set entries from all lmAction tables. */ + for ( StateList::Iter st = graph->stateList; st.lte(); st++ ) { + for ( TransList::Iter trans = st->outList; trans.lte(); trans++ ) { + if ( trans->lmActionTable.length() > 0 ) { + LmActionTableEl *lmAct = trans->lmActionTable.data; + StateAp *toState = trans->toState; + assert( toState ); + + /* Can only optimize this if there are no transitions out. + * Note there can be out transitions going nowhere with + * actions and they too must inhibit this optimization. */ + if ( toState->outList.length() > 0 ) { + /* Fill the item sets. */ + graph->markReachableFromHereStopFinal( toState ); + for ( StateList::Iter ms = graph->stateList; ms.lte(); ms++ ) { + if ( ms->stateBits & STB_ISMARKED ) { + ms->lmItemSet.insert( lmAct->value ); + ms->stateBits &= ~ STB_ISMARKED; + } + } + } + } + } + } + + /* The lmItem sets are now filled, telling us which longest match rules + * can succeed in which states. First determine if we need to make sure + * act is defaulted to zero. We need to do this if there are any states + * with lmItemSet.length() > 1 and NULL is included. That is, that the + * switch may get called when in fact nothing has been matched. */ + int maxItemSetLength = 0; + graph->markReachableFromHereStopFinal( graph->startState ); + for ( StateList::Iter ms = graph->stateList; ms.lte(); ms++ ) { + if ( ms->stateBits & STB_ISMARKED ) { + if ( ms->lmItemSet.length() > maxItemSetLength ) + maxItemSetLength = ms->lmItemSet.length(); + ms->stateBits &= ~ STB_ISMARKED; + } + } + + /* The actions executed on starting to match a token. */ + graph->isolateStartState(); + graph->startState->toStateActionTable.setAction( pd->initTokStartOrd, pd->initTokStart ); + graph->startState->fromStateActionTable.setAction( pd->setTokStartOrd, pd->setTokStart ); + if ( maxItemSetLength > 1 ) { + /* The longest match action switch may be called when tokens are + * matched, in which case act must be initialized, there must be a + * case to handle the error, and the generated machine will require an + * error state. */ + lmSwitchHandlesError = true; + pd->lmRequiresErrorState = true; + graph->startState->toStateActionTable.setAction( pd->initActIdOrd, pd->initActId ); + } + + /* The place to store transitions to restart. It maybe possible for the + * restarting to affect the searching through the graph that follows. For + * now take the safe route and save the list of transitions to restart + * until after all searching is done. */ + Vector<TransAp*> restartTrans; + + /* Set actions that do immediate token recognition, set the longest match part + * id and set the token ending. */ + for ( StateList::Iter st = graph->stateList; st.lte(); st++ ) { + for ( TransList::Iter trans = st->outList; trans.lte(); trans++ ) { + if ( trans->lmActionTable.length() > 0 ) { + LmActionTableEl *lmAct = trans->lmActionTable.data; + StateAp *toState = trans->toState; + assert( toState ); + + /* Can only optimize this if there are no transitions out. + * Note there can be out transitions going nowhere with + * actions and they too must inhibit this optimization. */ + if ( toState->outList.length() == 0 ) { + /* Can execute the immediate action for the longest match + * part. Redirect the action to the start state. + * + * NOTE: When we need to inhibit on_last due to leaving + * actions the above test suffices. If the state has out + * actions then it will fail because the out action will + * have been transferred to an error transition, which + * makes the outlist non-empty. */ + trans->actionTable.setAction( lmAct->key, + lmAct->value->actOnLast ); + restartTrans.append( trans ); + } + else { + /* Look for non final states that have a non-empty item + * set. If these are present then we need to record the + * end of the token. Also Find the highest item set + * length reachable from here (excluding at transtions to + * final states). */ + bool nonFinalNonEmptyItemSet = false; + maxItemSetLength = 0; + graph->markReachableFromHereStopFinal( toState ); + for ( StateList::Iter ms = graph->stateList; ms.lte(); ms++ ) { + if ( ms->stateBits & STB_ISMARKED ) { + if ( ms->lmItemSet.length() > 0 && !ms->isFinState() ) + nonFinalNonEmptyItemSet = true; + if ( ms->lmItemSet.length() > maxItemSetLength ) + maxItemSetLength = ms->lmItemSet.length(); + ms->stateBits &= ~ STB_ISMARKED; + } + } + + /* If there are reachable states that are not final and + * have non empty item sets or that have an item set + * length greater than one then we need to set tokend + * because the error action that matches the token will + * require it. */ + if ( nonFinalNonEmptyItemSet || maxItemSetLength > 1 ) + trans->actionTable.setAction( pd->setTokEndOrd, pd->setTokEnd ); + + /* Some states may not know which longest match item to + * execute, must set it. */ + if ( maxItemSetLength > 1 ) { + /* There are transitions out, another match may come. */ + trans->actionTable.setAction( lmAct->key, + lmAct->value->setActId ); + } + } + } + } + } + + /* Now that all graph searching is done it certainly safe set the + * restarting. It may be safe above, however this must be verified. */ + for ( Vector<TransAp*>::Iter pt = restartTrans; pt.lte(); pt++ ) + restart( graph, *pt ); + + int lmErrActionOrd = pd->curActionOrd++; + + /* Embed the error for recognizing a char. */ + for ( StateList::Iter st = graph->stateList; st.lte(); st++ ) { + if ( st->lmItemSet.length() == 1 && st->lmItemSet[0] != 0 ) { + if ( st->isFinState() ) { + /* On error execute the onActNext action, which knows that + * the last character of the token was one back and restart. */ + graph->setErrorTarget( st, graph->startState, &lmErrActionOrd, + &st->lmItemSet[0]->actOnNext, 1 ); + st->eofActionTable.setAction( lmErrActionOrd, + st->lmItemSet[0]->actOnNext ); + st->eofTarget = graph->startState; + } + else { + graph->setErrorTarget( st, graph->startState, &lmErrActionOrd, + &st->lmItemSet[0]->actLagBehind, 1 ); + st->eofActionTable.setAction( lmErrActionOrd, + st->lmItemSet[0]->actLagBehind ); + st->eofTarget = graph->startState; + } + } + else if ( st->lmItemSet.length() > 1 ) { + /* Need to use the select. Take note of which items the select + * is needed for so only the necessary actions are included. */ + for ( LmItemSet::Iter plmi = st->lmItemSet; plmi.lte(); plmi++ ) { + if ( *plmi != 0 ) + (*plmi)->inLmSelect = true; + } + /* On error, execute the action select and go to the start state. */ + graph->setErrorTarget( st, graph->startState, &lmErrActionOrd, + &lmActSelect, 1 ); + st->eofActionTable.setAction( lmErrActionOrd, lmActSelect ); + st->eofTarget = graph->startState; + } + } + + /* Finally, the start state should be made final. */ + graph->setFinState( graph->startState ); +} + +void LongestMatch::transferScannerLeavingActions( FsmAp *graph ) +{ + for ( StateList::Iter st = graph->stateList; st.lte(); st++ ) { + if ( st->outActionTable.length() > 0 ) + graph->setErrorActions( st, st->outActionTable ); + } +} + +FsmAp *LongestMatch::walk( ParseData *pd ) +{ + /* The longest match has it's own name scope. */ + NameFrame nameFrame = pd->enterNameScope( true, 1 ); + + /* Make each part of the longest match. */ + FsmAp **parts = new FsmAp*[longestMatchList->length()]; + LmPartList::Iter lmi = *longestMatchList; + for ( int i = 0; lmi.lte(); lmi++, i++ ) { + /* Create the machine and embed the setting of the longest match id. */ + parts[i] = lmi->join->walk( pd ); + parts[i]->longMatchAction( pd->curActionOrd++, lmi ); + } + + /* Before we union the patterns we need to deal with leaving actions. They + * are transfered to error transitions out of the final states (like local + * error actions) and to eof actions. In the scanner we need to forbid + * on_last for any final state that has an leaving action. */ + for ( int i = 0; i < longestMatchList->length(); i++ ) + transferScannerLeavingActions( parts[i] ); + + /* Union machines one and up with machine zero. The grammar dictates that + * there will always be at least one part. */ + FsmAp *rtnVal = parts[0]; + for ( int i = 1; i < longestMatchList->length(); i++ ) { + rtnVal->unionOp( parts[i] ); + afterOpMinimize( rtnVal ); + } + + runLongestMatch( pd, rtnVal ); + + /* Pop the name scope. */ + pd->popNameScope( nameFrame ); + + delete[] parts; + return rtnVal; +} + +FsmAp *MachineDef::walk( ParseData *pd ) +{ + FsmAp *rtnVal = 0; + switch ( type ) { + case JoinType: + rtnVal = join->walk( pd ); + break; + case LongestMatchType: + rtnVal = longestMatch->walk( pd ); + break; + case LengthDefType: + condData->lastCondKey.increment(); + rtnVal = new FsmAp(); + rtnVal->concatFsm( condData->lastCondKey ); + break; + } + return rtnVal; +} + +void MachineDef::makeNameTree( ParseData *pd ) +{ + switch ( type ) { + case JoinType: + join->makeNameTree( pd ); + break; + case LongestMatchType: + longestMatch->makeNameTree( pd ); + break; + case LengthDefType: + break; + } +} + +void MachineDef::resolveNameRefs( ParseData *pd ) +{ + switch ( type ) { + case JoinType: + join->resolveNameRefs( pd ); + break; + case LongestMatchType: + longestMatch->resolveNameRefs( pd ); + break; + case LengthDefType: + break; + } +} + + +/* Construct with a location and the first expression. */ +Join::Join( const InputLoc &loc, Expression *expr ) +: + loc(loc) +{ + exprList.append( expr ); +} + +/* Construct with a location and the first expression. */ +Join::Join( Expression *expr ) +{ + exprList.append( expr ); +} + +/* Walk an expression node. */ +FsmAp *Join::walk( ParseData *pd ) +{ + if ( exprList.length() > 1 ) + return walkJoin( pd ); + else + return exprList.head->walk( pd ); +} + +/* There is a list of expressions to join. */ +FsmAp *Join::walkJoin( ParseData *pd ) +{ + /* We enter into a new name scope. */ + NameFrame nameFrame = pd->enterNameScope( true, 1 ); + + /* Evaluate the machines. */ + FsmAp **fsms = new FsmAp*[exprList.length()]; + ExprList::Iter expr = exprList; + for ( int e = 0; e < exprList.length(); e++, expr++ ) + fsms[e] = expr->walk( pd ); + + /* Get the start and final names. Final is + * guaranteed to exist, start is not. */ + NameInst *startName = pd->curNameInst->start; + NameInst *finalName = pd->curNameInst->final; + + int startId = -1; + if ( startName != 0 ) { + /* Take note that there was an implicit link to the start machine. */ + pd->localNameScope->referencedNames.append( startName ); + startId = startName->id; + } + + /* A final id of -1 indicates there is no epsilon that references the + * final state, therefor do not create one or set an entry point to it. */ + int finalId = -1; + if ( finalName->numRefs > 0 ) + finalId = finalName->id; + + /* Join machines 1 and up onto machine 0. */ + FsmAp *retFsm = fsms[0]; + retFsm->joinOp( startId, finalId, fsms+1, exprList.length()-1 ); + + /* We can now unset entry points that are not longer used. */ + pd->unsetObsoleteEntries( retFsm ); + + /* Pop the name scope. */ + pd->popNameScope( nameFrame ); + + delete[] fsms; + return retFsm; +} + +void Join::makeNameTree( ParseData *pd ) +{ + if ( exprList.length() > 1 ) { + /* Create the new anonymous scope. */ + NameInst *prevNameInst = pd->curNameInst; + pd->curNameInst = pd->addNameInst( loc, 0, false ); + + /* Join scopes need an implicit "final" target. */ + pd->curNameInst->final = new NameInst( InputLoc(), pd->curNameInst, "final", + pd->nextNameId++, false ); + + /* Recurse into all expressions in the list. */ + for ( ExprList::Iter expr = exprList; expr.lte(); expr++ ) + expr->makeNameTree( pd ); + + /* The name scope ends, pop the name instantiation. */ + pd->curNameInst = prevNameInst; + } + else { + /* Recurse into the single expression. */ + exprList.head->makeNameTree( pd ); + } +} + + +void Join::resolveNameRefs( ParseData *pd ) +{ + /* Branch on whether or not there is to be a join. */ + if ( exprList.length() > 1 ) { + /* The variable definition enters a new scope. */ + NameFrame nameFrame = pd->enterNameScope( true, 1 ); + + /* The join scope must contain a start label. */ + NameSet resolved = pd->resolvePart( pd->localNameScope, "start", true ); + if ( resolved.length() > 0 ) { + /* Take the first. */ + pd->curNameInst->start = resolved[0]; + if ( resolved.length() > 1 ) { + /* Complain about the multiple references. */ + error(loc) << "join operation has multiple start labels" << endl; + errorStateLabels( resolved ); + } + } + + /* Make sure there is a start label. */ + if ( pd->curNameInst->start != 0 ) { + /* There is an implicit reference to start name. */ + pd->curNameInst->start->numRefs += 1; + } + else { + /* No start label. */ + error(loc) << "join operation has no start label" << endl; + } + + /* Recurse into all expressions in the list. */ + for ( ExprList::Iter expr = exprList; expr.lte(); expr++ ) + expr->resolveNameRefs( pd ); + + /* The name scope ends, pop the name instantiation. */ + pd->popNameScope( nameFrame ); + } + else { + /* Recurse into the single expression. */ + exprList.head->resolveNameRefs( pd ); + } +} + +/* Clean up after an expression node. */ +Expression::~Expression() +{ + switch ( type ) { + case OrType: case IntersectType: case SubtractType: + case StrongSubtractType: + delete expression; + delete term; + break; + case TermType: + delete term; + break; + case BuiltinType: + break; + } +} + +/* Evaluate a single expression node. */ +FsmAp *Expression::walk( ParseData *pd, bool lastInSeq ) +{ + FsmAp *rtnVal = 0; + switch ( type ) { + case OrType: { + /* Evaluate the expression. */ + rtnVal = expression->walk( pd, false ); + /* Evaluate the term. */ + FsmAp *rhs = term->walk( pd ); + /* Perform union. */ + rtnVal->unionOp( rhs ); + afterOpMinimize( rtnVal, lastInSeq ); + break; + } + case IntersectType: { + /* Evaluate the expression. */ + rtnVal = expression->walk( pd ); + /* Evaluate the term. */ + FsmAp *rhs = term->walk( pd ); + /* Perform intersection. */ + rtnVal->intersectOp( rhs ); + afterOpMinimize( rtnVal, lastInSeq ); + break; + } + case SubtractType: { + /* Evaluate the expression. */ + rtnVal = expression->walk( pd ); + /* Evaluate the term. */ + FsmAp *rhs = term->walk( pd ); + /* Perform subtraction. */ + rtnVal->subtractOp( rhs ); + afterOpMinimize( rtnVal, lastInSeq ); + break; + } + case StrongSubtractType: { + /* Evaluate the expression. */ + rtnVal = expression->walk( pd ); + + /* Evaluate the term and pad it with any* machines. */ + FsmAp *rhs = dotStarFsm( pd ); + FsmAp *termFsm = term->walk( pd ); + FsmAp *trailAnyStar = dotStarFsm( pd ); + rhs->concatOp( termFsm ); + rhs->concatOp( trailAnyStar ); + + /* Perform subtraction. */ + rtnVal->subtractOp( rhs ); + afterOpMinimize( rtnVal, lastInSeq ); + break; + } + case TermType: { + /* Return result of the term. */ + rtnVal = term->walk( pd ); + break; + } + case BuiltinType: { + /* Duplicate the builtin. */ + rtnVal = makeBuiltin( builtin, pd ); + break; + } + } + + return rtnVal; +} + +void Expression::makeNameTree( ParseData *pd ) +{ + switch ( type ) { + case OrType: + case IntersectType: + case SubtractType: + case StrongSubtractType: + expression->makeNameTree( pd ); + term->makeNameTree( pd ); + break; + case TermType: + term->makeNameTree( pd ); + break; + case BuiltinType: + break; + } +} + +void Expression::resolveNameRefs( ParseData *pd ) +{ + switch ( type ) { + case OrType: + case IntersectType: + case SubtractType: + case StrongSubtractType: + expression->resolveNameRefs( pd ); + term->resolveNameRefs( pd ); + break; + case TermType: + term->resolveNameRefs( pd ); + break; + case BuiltinType: + break; + } +} + +/* Clean up after a term node. */ +Term::~Term() +{ + switch ( type ) { + case ConcatType: + case RightStartType: + case RightFinishType: + case LeftType: + delete term; + delete factorWithAug; + break; + case FactorWithAugType: + delete factorWithAug; + break; + } +} + +/* Evaluate a term node. */ +FsmAp *Term::walk( ParseData *pd, bool lastInSeq ) +{ + FsmAp *rtnVal = 0; + switch ( type ) { + case ConcatType: { + /* Evaluate the Term. */ + rtnVal = term->walk( pd, false ); + /* Evaluate the FactorWithRep. */ + FsmAp *rhs = factorWithAug->walk( pd ); + /* Perform concatenation. */ + rtnVal->concatOp( rhs ); + afterOpMinimize( rtnVal, lastInSeq ); + break; + } + case RightStartType: { + /* Evaluate the Term. */ + rtnVal = term->walk( pd ); + + /* Evaluate the FactorWithRep. */ + FsmAp *rhs = factorWithAug->walk( pd ); + + /* Set up the priority descriptors. The left machine gets the + * lower priority where as the right get the higher start priority. */ + priorDescs[0].key = pd->nextPriorKey++; + priorDescs[0].priority = 0; + rtnVal->allTransPrior( pd->curPriorOrd++, &priorDescs[0] ); + + /* The start transitions of the right machine gets the higher + * priority. Use the same unique key. */ + priorDescs[1].key = priorDescs[0].key; + priorDescs[1].priority = 1; + rhs->startFsmPrior( pd->curPriorOrd++, &priorDescs[1] ); + + /* Perform concatenation. */ + rtnVal->concatOp( rhs ); + afterOpMinimize( rtnVal, lastInSeq ); + break; + } + case RightFinishType: { + /* Evaluate the Term. */ + rtnVal = term->walk( pd ); + + /* Evaluate the FactorWithRep. */ + FsmAp *rhs = factorWithAug->walk( pd ); + + /* Set up the priority descriptors. The left machine gets the + * lower priority where as the finishing transitions to the right + * get the higher priority. */ + priorDescs[0].key = pd->nextPriorKey++; + priorDescs[0].priority = 0; + rtnVal->allTransPrior( pd->curPriorOrd++, &priorDescs[0] ); + + /* The finishing transitions of the right machine get the higher + * priority. Use the same unique key. */ + priorDescs[1].key = priorDescs[0].key; + priorDescs[1].priority = 1; + rhs->finishFsmPrior( pd->curPriorOrd++, &priorDescs[1] ); + + /* If the right machine's start state is final we need to guard + * against the left machine persisting by moving through the empty + * string. */ + if ( rhs->startState->isFinState() ) { + rhs->startState->outPriorTable.setPrior( + pd->curPriorOrd++, &priorDescs[1] ); + } + + /* Perform concatenation. */ + rtnVal->concatOp( rhs ); + afterOpMinimize( rtnVal, lastInSeq ); + break; + } + case LeftType: { + /* Evaluate the Term. */ + rtnVal = term->walk( pd ); + + /* Evaluate the FactorWithRep. */ + FsmAp *rhs = factorWithAug->walk( pd ); + + /* Set up the priority descriptors. The left machine gets the + * higher priority. */ + priorDescs[0].key = pd->nextPriorKey++; + priorDescs[0].priority = 1; + rtnVal->allTransPrior( pd->curPriorOrd++, &priorDescs[0] ); + + /* The right machine gets the lower priority. We cannot use + * allTransPrior here in case the start state of the right machine + * is final. It would allow the right machine thread to run along + * with the left if just passing through the start state. Using + * startFsmPrior prevents this. */ + priorDescs[1].key = priorDescs[0].key; + priorDescs[1].priority = 0; + rhs->startFsmPrior( pd->curPriorOrd++, &priorDescs[1] ); + + /* Perform concatenation. */ + rtnVal->concatOp( rhs ); + afterOpMinimize( rtnVal, lastInSeq ); + break; + } + case FactorWithAugType: { + rtnVal = factorWithAug->walk( pd ); + break; + } + } + return rtnVal; +} + +void Term::makeNameTree( ParseData *pd ) +{ + switch ( type ) { + case ConcatType: + case RightStartType: + case RightFinishType: + case LeftType: + term->makeNameTree( pd ); + factorWithAug->makeNameTree( pd ); + break; + case FactorWithAugType: + factorWithAug->makeNameTree( pd ); + break; + } +} + +void Term::resolveNameRefs( ParseData *pd ) +{ + switch ( type ) { + case ConcatType: + case RightStartType: + case RightFinishType: + case LeftType: + term->resolveNameRefs( pd ); + factorWithAug->resolveNameRefs( pd ); + break; + case FactorWithAugType: + factorWithAug->resolveNameRefs( pd ); + break; + } +} + +/* Clean up after a factor with augmentation node. */ +FactorWithAug::~FactorWithAug() +{ + delete factorWithRep; + + /* Walk the vector of parser actions, deleting function names. */ + + /* Clean up priority descriptors. */ + if ( priorDescs != 0 ) + delete[] priorDescs; +} + +void FactorWithAug::assignActions( ParseData *pd, FsmAp *graph, int *actionOrd ) +{ + /* Assign actions. */ + for ( int i = 0; i < actions.length(); i++ ) { + switch ( actions[i].type ) { + /* Transition actions. */ + case at_start: + graph->startFsmAction( actionOrd[i], actions[i].action ); + afterOpMinimize( graph ); + break; + case at_all: + graph->allTransAction( actionOrd[i], actions[i].action ); + break; + case at_finish: + graph->finishFsmAction( actionOrd[i], actions[i].action ); + break; + case at_leave: + graph->leaveFsmAction( actionOrd[i], actions[i].action ); + break; + + /* Global error actions. */ + case at_start_gbl_error: + graph->startErrorAction( actionOrd[i], actions[i].action, 0 ); + afterOpMinimize( graph ); + break; + case at_all_gbl_error: + graph->allErrorAction( actionOrd[i], actions[i].action, 0 ); + break; + case at_final_gbl_error: + graph->finalErrorAction( actionOrd[i], actions[i].action, 0 ); + break; + case at_not_start_gbl_error: + graph->notStartErrorAction( actionOrd[i], actions[i].action, 0 ); + break; + case at_not_final_gbl_error: + graph->notFinalErrorAction( actionOrd[i], actions[i].action, 0 ); + break; + case at_middle_gbl_error: + graph->middleErrorAction( actionOrd[i], actions[i].action, 0 ); + break; + + /* Local error actions. */ + case at_start_local_error: + graph->startErrorAction( actionOrd[i], actions[i].action, + actions[i].localErrKey ); + afterOpMinimize( graph ); + break; + case at_all_local_error: + graph->allErrorAction( actionOrd[i], actions[i].action, + actions[i].localErrKey ); + break; + case at_final_local_error: + graph->finalErrorAction( actionOrd[i], actions[i].action, + actions[i].localErrKey ); + break; + case at_not_start_local_error: + graph->notStartErrorAction( actionOrd[i], actions[i].action, + actions[i].localErrKey ); + break; + case at_not_final_local_error: + graph->notFinalErrorAction( actionOrd[i], actions[i].action, + actions[i].localErrKey ); + break; + case at_middle_local_error: + graph->middleErrorAction( actionOrd[i], actions[i].action, + actions[i].localErrKey ); + break; + + /* EOF actions. */ + case at_start_eof: + graph->startEOFAction( actionOrd[i], actions[i].action ); + afterOpMinimize( graph ); + break; + case at_all_eof: + graph->allEOFAction( actionOrd[i], actions[i].action ); + break; + case at_final_eof: + graph->finalEOFAction( actionOrd[i], actions[i].action ); + break; + case at_not_start_eof: + graph->notStartEOFAction( actionOrd[i], actions[i].action ); + break; + case at_not_final_eof: + graph->notFinalEOFAction( actionOrd[i], actions[i].action ); + break; + case at_middle_eof: + graph->middleEOFAction( actionOrd[i], actions[i].action ); + break; + + /* To State Actions. */ + case at_start_to_state: + graph->startToStateAction( actionOrd[i], actions[i].action ); + afterOpMinimize( graph ); + break; + case at_all_to_state: + graph->allToStateAction( actionOrd[i], actions[i].action ); + break; + case at_final_to_state: + graph->finalToStateAction( actionOrd[i], actions[i].action ); + break; + case at_not_start_to_state: + graph->notStartToStateAction( actionOrd[i], actions[i].action ); + break; + case at_not_final_to_state: + graph->notFinalToStateAction( actionOrd[i], actions[i].action ); + break; + case at_middle_to_state: + graph->middleToStateAction( actionOrd[i], actions[i].action ); + break; + + /* From State Actions. */ + case at_start_from_state: + graph->startFromStateAction( actionOrd[i], actions[i].action ); + afterOpMinimize( graph ); + break; + case at_all_from_state: + graph->allFromStateAction( actionOrd[i], actions[i].action ); + break; + case at_final_from_state: + graph->finalFromStateAction( actionOrd[i], actions[i].action ); + break; + case at_not_start_from_state: + graph->notStartFromStateAction( actionOrd[i], actions[i].action ); + break; + case at_not_final_from_state: + graph->notFinalFromStateAction( actionOrd[i], actions[i].action ); + break; + case at_middle_from_state: + graph->middleFromStateAction( actionOrd[i], actions[i].action ); + break; + + /* Remaining cases, prevented by the parser. */ + default: + assert( false ); + break; + } + } +} + +void FactorWithAug::assignPriorities( FsmAp *graph, int *priorOrd ) +{ + /* Assign priorities. */ + for ( int i = 0; i < priorityAugs.length(); i++ ) { + switch ( priorityAugs[i].type ) { + case at_start: + graph->startFsmPrior( priorOrd[i], &priorDescs[i]); + /* Start fsm priorities are a special case that may require + * minimization afterwards. */ + afterOpMinimize( graph ); + break; + case at_all: + graph->allTransPrior( priorOrd[i], &priorDescs[i] ); + break; + case at_finish: + graph->finishFsmPrior( priorOrd[i], &priorDescs[i] ); + break; + case at_leave: + graph->leaveFsmPrior( priorOrd[i], &priorDescs[i] ); + break; + + default: + /* Parser Prevents this case. */ + break; + } + } +} + +void FactorWithAug::assignConditions( FsmAp *graph ) +{ + for ( int i = 0; i < conditions.length(); i++ ) { + switch ( conditions[i].type ) { + /* Transition actions. */ + case at_start: + graph->startFsmCondition( conditions[i].action, conditions[i].sense ); + afterOpMinimize( graph ); + break; + case at_all: + graph->allTransCondition( conditions[i].action, conditions[i].sense ); + break; + case at_leave: + graph->leaveFsmCondition( conditions[i].action, conditions[i].sense ); + break; + default: + break; + } + } +} + + +/* Evaluate a factor with augmentation node. */ +FsmAp *FactorWithAug::walk( ParseData *pd ) +{ + /* Enter into the scopes created for the labels. */ + NameFrame nameFrame = pd->enterNameScope( false, labels.length() ); + + /* Make the array of function orderings. */ + int *actionOrd = 0; + if ( actions.length() > 0 ) + actionOrd = new int[actions.length()]; + + /* First walk the list of actions, assigning order to all starting + * actions. */ + for ( int i = 0; i < actions.length(); i++ ) { + if ( actions[i].type == at_start || + actions[i].type == at_start_gbl_error || + actions[i].type == at_start_local_error || + actions[i].type == at_start_to_state || + actions[i].type == at_start_from_state || + actions[i].type == at_start_eof ) + actionOrd[i] = pd->curActionOrd++; + } + + /* Evaluate the factor with repetition. */ + FsmAp *rtnVal = factorWithRep->walk( pd ); + + /* Compute the remaining action orderings. */ + for ( int i = 0; i < actions.length(); i++ ) { + if ( actions[i].type != at_start && + actions[i].type != at_start_gbl_error && + actions[i].type != at_start_local_error && + actions[i].type != at_start_to_state && + actions[i].type != at_start_from_state && + actions[i].type != at_start_eof ) + actionOrd[i] = pd->curActionOrd++; + } + + /* Embed conditions. */ + assignConditions( rtnVal ); + + /* Embed actions. */ + assignActions( pd, rtnVal , actionOrd ); + + /* Make the array of priority orderings. Orderings are local to this walk + * of the factor with augmentation. */ + int *priorOrd = 0; + if ( priorityAugs.length() > 0 ) + priorOrd = new int[priorityAugs.length()]; + + /* Walk all priorities, assigning the priority ordering. */ + for ( int i = 0; i < priorityAugs.length(); i++ ) + priorOrd[i] = pd->curPriorOrd++; + + /* If the priority descriptors have not been made, make them now. Make + * priority descriptors for each priority asignment that will be passed to + * the fsm. Used to keep track of the key, value and used bit. */ + if ( priorDescs == 0 && priorityAugs.length() > 0 ) { + priorDescs = new PriorDesc[priorityAugs.length()]; + for ( int i = 0; i < priorityAugs.length(); i++ ) { + /* Init the prior descriptor for the priority setting. */ + priorDescs[i].key = priorityAugs[i].priorKey; + priorDescs[i].priority = priorityAugs[i].priorValue; + } + } + + /* Assign priorities into the machine. */ + assignPriorities( rtnVal, priorOrd ); + + /* Assign epsilon transitions. */ + for ( int e = 0; e < epsilonLinks.length(); e++ ) { + /* Get the name, which may not exist. If it doesn't then silently + * ignore it because an error has already been reported. */ + NameInst *epTarg = pd->epsilonResolvedLinks[pd->nextEpsilonResolvedLink++]; + if ( epTarg != 0 ) { + /* Make the epsilon transitions. */ + rtnVal->epsilonTrans( epTarg->id ); + + /* Note that we have made a link to the name. */ + pd->localNameScope->referencedNames.append( epTarg ); + } + } + + /* Set entry points for labels. */ + if ( labels.length() > 0 ) { + /* Pop the names. */ + pd->resetNameScope( nameFrame ); + + /* Make labels that are referenced into entry points. */ + for ( int i = 0; i < labels.length(); i++ ) { + pd->enterNameScope( false, 1 ); + + /* Will always be found. */ + NameInst *name = pd->curNameInst; + + /* If the name is referenced then set the entry point. */ + if ( name->numRefs > 0 ) + rtnVal->setEntry( name->id, rtnVal->startState ); + } + + pd->popNameScope( nameFrame ); + } + + if ( priorOrd != 0 ) + delete[] priorOrd; + if ( actionOrd != 0 ) + delete[] actionOrd; + return rtnVal; +} + +void FactorWithAug::makeNameTree( ParseData *pd ) +{ + /* Add the labels to the tree of instantiated names. Each label + * makes a new scope. */ + NameInst *prevNameInst = pd->curNameInst; + for ( int i = 0; i < labels.length(); i++ ) + pd->curNameInst = pd->addNameInst( labels[i].loc, labels[i].data, true ); + + /* Recurse, then pop the names. */ + factorWithRep->makeNameTree( pd ); + pd->curNameInst = prevNameInst; +} + + +void FactorWithAug::resolveNameRefs( ParseData *pd ) +{ + /* Enter into the name scope created by any labels. */ + NameFrame nameFrame = pd->enterNameScope( false, labels.length() ); + + /* Note action references. */ + for ( int i = 0; i < actions.length(); i++ ) + actions[i].action->actionRefs.append( pd->localNameScope ); + + /* Recurse first. IMPORTANT: we must do the exact same traversal as when + * the tree is constructed. */ + factorWithRep->resolveNameRefs( pd ); + + /* Resolve epsilon transitions. */ + for ( int ep = 0; ep < epsilonLinks.length(); ep++ ) { + /* Get the link. */ + EpsilonLink &link = epsilonLinks[ep]; + NameInst *resolvedName = 0; + + if ( link.target.length() == 1 && strcmp( link.target.data[0], "final" ) == 0 ) { + /* Epsilon drawn to an implicit final state. An implicit final is + * only available in join operations. */ + resolvedName = pd->localNameScope->final; + } + else { + /* Do an search for the name. */ + NameSet resolved; + pd->resolveFrom( resolved, pd->localNameScope, link.target, 0 ); + if ( resolved.length() > 0 ) { + /* Take the first one. */ + resolvedName = resolved[0]; + if ( resolved.length() > 1 ) { + /* Complain about the multiple references. */ + error(link.loc) << "state reference " << link.target << + " resolves to multiple entry points" << endl; + errorStateLabels( resolved ); + } + } + } + + /* This is tricky, we stuff resolved epsilon transitions into one long + * vector in the parse data structure. Since the name resolution and + * graph generation both do identical walks of the parse tree we + * should always find the link resolutions in the right place. */ + pd->epsilonResolvedLinks.append( resolvedName ); + + if ( resolvedName != 0 ) { + /* Found the name, bump of the reference count on it. */ + resolvedName->numRefs += 1; + } + else { + /* Complain, no recovery action, the epsilon op will ignore any + * epsilon transitions whose names did not resolve. */ + error(link.loc) << "could not resolve label " << link.target << endl; + } + } + + if ( labels.length() > 0 ) + pd->popNameScope( nameFrame ); +} + + +/* Clean up after a factor with repetition node. */ +FactorWithRep::~FactorWithRep() +{ + switch ( type ) { + case StarType: case StarStarType: case OptionalType: case PlusType: + case ExactType: case MaxType: case MinType: case RangeType: + delete factorWithRep; + break; + case FactorWithNegType: + delete factorWithNeg; + break; + } +} + +/* Evaluate a factor with repetition node. */ +FsmAp *FactorWithRep::walk( ParseData *pd ) +{ + FsmAp *retFsm = 0; + + switch ( type ) { + case StarType: { + /* Evaluate the FactorWithRep. */ + retFsm = factorWithRep->walk( pd ); + if ( retFsm->startState->isFinState() ) { + warning(loc) << "applying kleene star to a machine that " + "accepts zero length word" << endl; + retFsm->unsetFinState( retFsm->startState ); + } + + /* Shift over the start action orders then do the kleene star. */ + pd->curActionOrd += retFsm->shiftStartActionOrder( pd->curActionOrd ); + retFsm->starOp( ); + afterOpMinimize( retFsm ); + break; + } + case StarStarType: { + /* Evaluate the FactorWithRep. */ + retFsm = factorWithRep->walk( pd ); + if ( retFsm->startState->isFinState() ) { + warning(loc) << "applying kleene star to a machine that " + "accepts zero length word" << endl; + } + + /* Set up the prior descs. All gets priority one, whereas leaving gets + * priority zero. Make a unique key so that these priorities don't + * interfere with any priorities set by the user. */ + priorDescs[0].key = pd->nextPriorKey++; + priorDescs[0].priority = 1; + retFsm->allTransPrior( pd->curPriorOrd++, &priorDescs[0] ); + + /* Leaveing gets priority 0. Use same unique key. */ + priorDescs[1].key = priorDescs[0].key; + priorDescs[1].priority = 0; + retFsm->leaveFsmPrior( pd->curPriorOrd++, &priorDescs[1] ); + + /* Shift over the start action orders then do the kleene star. */ + pd->curActionOrd += retFsm->shiftStartActionOrder( pd->curActionOrd ); + retFsm->starOp( ); + afterOpMinimize( retFsm ); + break; + } + case OptionalType: { + /* Make the null fsm. */ + FsmAp *nu = new FsmAp(); + nu->lambdaFsm( ); + + /* Evaluate the FactorWithRep. */ + retFsm = factorWithRep->walk( pd ); + + /* Perform the question operator. */ + retFsm->unionOp( nu ); + afterOpMinimize( retFsm ); + break; + } + case PlusType: { + /* Evaluate the FactorWithRep. */ + retFsm = factorWithRep->walk( pd ); + if ( retFsm->startState->isFinState() ) { + warning(loc) << "applying plus operator to a machine that " + "accepts zero length word" << endl; + } + + /* Need a duplicated for the star end. */ + FsmAp *dup = new FsmAp( *retFsm ); + + /* The start func orders need to be shifted before doing the star. */ + pd->curActionOrd += dup->shiftStartActionOrder( pd->curActionOrd ); + + /* Star the duplicate. */ + dup->starOp( ); + afterOpMinimize( dup ); + + retFsm->concatOp( dup ); + afterOpMinimize( retFsm ); + break; + } + case ExactType: { + /* Get an int from the repetition amount. */ + if ( lowerRep == 0 ) { + /* No copies. Don't need to evaluate the factorWithRep. + * This Defeats the purpose so give a warning. */ + warning(loc) << "exactly zero repetitions results " + "in the null machine" << endl; + + retFsm = new FsmAp(); + retFsm->lambdaFsm(); + } + else { + /* Evaluate the first FactorWithRep. */ + retFsm = factorWithRep->walk( pd ); + if ( retFsm->startState->isFinState() ) { + warning(loc) << "applying repetition to a machine that " + "accepts zero length word" << endl; + } + + /* The start func orders need to be shifted before doing the + * repetition. */ + pd->curActionOrd += retFsm->shiftStartActionOrder( pd->curActionOrd ); + + /* Do the repetition on the machine. Already guarded against n == 0 */ + retFsm->repeatOp( lowerRep ); + afterOpMinimize( retFsm ); + } + break; + } + case MaxType: { + /* Get an int from the repetition amount. */ + if ( upperRep == 0 ) { + /* No copies. Don't need to evaluate the factorWithRep. + * This Defeats the purpose so give a warning. */ + warning(loc) << "max zero repetitions results " + "in the null machine" << endl; + + retFsm = new FsmAp(); + retFsm->lambdaFsm(); + } + else { + /* Evaluate the first FactorWithRep. */ + retFsm = factorWithRep->walk( pd ); + if ( retFsm->startState->isFinState() ) { + warning(loc) << "applying max repetition to a machine that " + "accepts zero length word" << endl; + } + + /* The start func orders need to be shifted before doing the + * repetition. */ + pd->curActionOrd += retFsm->shiftStartActionOrder( pd->curActionOrd ); + + /* Do the repetition on the machine. Already guarded against n == 0 */ + retFsm->optionalRepeatOp( upperRep ); + afterOpMinimize( retFsm ); + } + break; + } + case MinType: { + /* Evaluate the repeated machine. */ + retFsm = factorWithRep->walk( pd ); + if ( retFsm->startState->isFinState() ) { + warning(loc) << "applying min repetition to a machine that " + "accepts zero length word" << endl; + } + + /* The start func orders need to be shifted before doing the repetition + * and the kleene star. */ + pd->curActionOrd += retFsm->shiftStartActionOrder( pd->curActionOrd ); + + if ( lowerRep == 0 ) { + /* Acts just like a star op on the machine to return. */ + retFsm->starOp( ); + afterOpMinimize( retFsm ); + } + else { + /* Take a duplicate for the plus. */ + FsmAp *dup = new FsmAp( *retFsm ); + + /* Do repetition on the first half. */ + retFsm->repeatOp( lowerRep ); + afterOpMinimize( retFsm ); + + /* Star the duplicate. */ + dup->starOp( ); + afterOpMinimize( dup ); + + /* Tak on the kleene star. */ + retFsm->concatOp( dup ); + afterOpMinimize( retFsm ); + } + break; + } + case RangeType: { + /* Check for bogus range. */ + if ( upperRep - lowerRep < 0 ) { + error(loc) << "invalid range repetition" << endl; + + /* Return null machine as recovery. */ + retFsm = new FsmAp(); + retFsm->lambdaFsm(); + } + else if ( lowerRep == 0 && upperRep == 0 ) { + /* No copies. Don't need to evaluate the factorWithRep. This + * defeats the purpose so give a warning. */ + warning(loc) << "zero to zero repetitions results " + "in the null machine" << endl; + + retFsm = new FsmAp(); + retFsm->lambdaFsm(); + } + else { + /* Now need to evaluate the repeated machine. */ + retFsm = factorWithRep->walk( pd ); + if ( retFsm->startState->isFinState() ) { + warning(loc) << "applying range repetition to a machine that " + "accepts zero length word" << endl; + } + + /* The start func orders need to be shifted before doing both kinds + * of repetition. */ + pd->curActionOrd += retFsm->shiftStartActionOrder( pd->curActionOrd ); + + if ( lowerRep == 0 ) { + /* Just doing max repetition. Already guarded against n == 0. */ + retFsm->optionalRepeatOp( upperRep ); + afterOpMinimize( retFsm ); + } + else if ( lowerRep == upperRep ) { + /* Just doing exact repetition. Already guarded against n == 0. */ + retFsm->repeatOp( lowerRep ); + afterOpMinimize( retFsm ); + } + else { + /* This is the case that 0 < lowerRep < upperRep. Take a + * duplicate for the optional repeat. */ + FsmAp *dup = new FsmAp( *retFsm ); + + /* Do repetition on the first half. */ + retFsm->repeatOp( lowerRep ); + afterOpMinimize( retFsm ); + + /* Do optional repetition on the second half. */ + dup->optionalRepeatOp( upperRep - lowerRep ); + afterOpMinimize( dup ); + + /* Tak on the duplicate machine. */ + retFsm->concatOp( dup ); + afterOpMinimize( retFsm ); + } + } + break; + } + case FactorWithNegType: { + /* Evaluate the Factor. Pass it up. */ + retFsm = factorWithNeg->walk( pd ); + break; + }} + return retFsm; +} + +void FactorWithRep::makeNameTree( ParseData *pd ) +{ + switch ( type ) { + case StarType: + case StarStarType: + case OptionalType: + case PlusType: + case ExactType: + case MaxType: + case MinType: + case RangeType: + factorWithRep->makeNameTree( pd ); + break; + case FactorWithNegType: + factorWithNeg->makeNameTree( pd ); + break; + } +} + +void FactorWithRep::resolveNameRefs( ParseData *pd ) +{ + switch ( type ) { + case StarType: + case StarStarType: + case OptionalType: + case PlusType: + case ExactType: + case MaxType: + case MinType: + case RangeType: + factorWithRep->resolveNameRefs( pd ); + break; + case FactorWithNegType: + factorWithNeg->resolveNameRefs( pd ); + break; + } +} + +/* Clean up after a factor with negation node. */ +FactorWithNeg::~FactorWithNeg() +{ + switch ( type ) { + case NegateType: + case CharNegateType: + delete factorWithNeg; + break; + case FactorType: + delete factor; + break; + } +} + +/* Evaluate a factor with negation node. */ +FsmAp *FactorWithNeg::walk( ParseData *pd ) +{ + FsmAp *retFsm = 0; + + switch ( type ) { + case NegateType: { + /* Evaluate the factorWithNeg. */ + FsmAp *toNegate = factorWithNeg->walk( pd ); + + /* Negation is subtract from dot-star. */ + retFsm = dotStarFsm( pd ); + retFsm->subtractOp( toNegate ); + afterOpMinimize( retFsm ); + break; + } + case CharNegateType: { + /* Evaluate the factorWithNeg. */ + FsmAp *toNegate = factorWithNeg->walk( pd ); + + /* CharNegation is subtract from dot. */ + retFsm = dotFsm( pd ); + retFsm->subtractOp( toNegate ); + afterOpMinimize( retFsm ); + break; + } + case FactorType: { + /* Evaluate the Factor. Pass it up. */ + retFsm = factor->walk( pd ); + break; + }} + return retFsm; +} + +void FactorWithNeg::makeNameTree( ParseData *pd ) +{ + switch ( type ) { + case NegateType: + case CharNegateType: + factorWithNeg->makeNameTree( pd ); + break; + case FactorType: + factor->makeNameTree( pd ); + break; + } +} + +void FactorWithNeg::resolveNameRefs( ParseData *pd ) +{ + switch ( type ) { + case NegateType: + case CharNegateType: + factorWithNeg->resolveNameRefs( pd ); + break; + case FactorType: + factor->resolveNameRefs( pd ); + break; + } +} + +/* Clean up after a factor node. */ +Factor::~Factor() +{ + switch ( type ) { + case LiteralType: + delete literal; + break; + case RangeType: + delete range; + break; + case OrExprType: + delete reItem; + break; + case RegExprType: + delete regExpr; + break; + case ReferenceType: + break; + case ParenType: + delete join; + break; + case LongestMatchType: + delete longestMatch; + break; + } +} + +/* Evaluate a factor node. */ +FsmAp *Factor::walk( ParseData *pd ) +{ + FsmAp *rtnVal = 0; + switch ( type ) { + case LiteralType: + rtnVal = literal->walk( pd ); + break; + case RangeType: + rtnVal = range->walk( pd ); + break; + case OrExprType: + rtnVal = reItem->walk( pd, 0 ); + break; + case RegExprType: + rtnVal = regExpr->walk( pd, 0 ); + break; + case ReferenceType: + rtnVal = varDef->walk( pd ); + break; + case ParenType: + rtnVal = join->walk( pd ); + break; + case LongestMatchType: + rtnVal = longestMatch->walk( pd ); + break; + } + + return rtnVal; +} + +void Factor::makeNameTree( ParseData *pd ) +{ + switch ( type ) { + case LiteralType: + case RangeType: + case OrExprType: + case RegExprType: + break; + case ReferenceType: + varDef->makeNameTree( loc, pd ); + break; + case ParenType: + join->makeNameTree( pd ); + break; + case LongestMatchType: + longestMatch->makeNameTree( pd ); + break; + } +} + +void Factor::resolveNameRefs( ParseData *pd ) +{ + switch ( type ) { + case LiteralType: + case RangeType: + case OrExprType: + case RegExprType: + break; + case ReferenceType: + varDef->resolveNameRefs( pd ); + break; + case ParenType: + join->resolveNameRefs( pd ); + break; + case LongestMatchType: + longestMatch->resolveNameRefs( pd ); + break; + } +} + +/* Clean up a range object. Must delete the two literals. */ +Range::~Range() +{ + delete lowerLit; + delete upperLit; +} + +/* Evaluate a range. Gets the lower an upper key and makes an fsm range. */ +FsmAp *Range::walk( ParseData *pd ) +{ + /* Construct and verify the suitability of the lower end of the range. */ + FsmAp *lowerFsm = lowerLit->walk( pd ); + if ( !lowerFsm->checkSingleCharMachine() ) { + error(lowerLit->token.loc) << + "bad range lower end, must be a single character" << endl; + } + + /* Construct and verify the upper end. */ + FsmAp *upperFsm = upperLit->walk( pd ); + if ( !upperFsm->checkSingleCharMachine() ) { + error(upperLit->token.loc) << + "bad range upper end, must be a single character" << endl; + } + + /* Grab the keys from the machines, then delete them. */ + Key lowKey = lowerFsm->startState->outList.head->lowKey; + Key highKey = upperFsm->startState->outList.head->lowKey; + delete lowerFsm; + delete upperFsm; + + /* Validate the range. */ + if ( lowKey > highKey ) { + /* Recover by setting upper to lower; */ + error(lowerLit->token.loc) << "lower end of range is greater then upper end" << endl; + highKey = lowKey; + } + + /* Return the range now that it is validated. */ + FsmAp *retFsm = new FsmAp(); + retFsm->rangeFsm( lowKey, highKey ); + return retFsm; +} + +/* Evaluate a literal object. */ +FsmAp *Literal::walk( ParseData *pd ) +{ + /* FsmAp to return, is the alphabet signed. */ + FsmAp *rtnVal = 0; + + switch ( type ) { + case Number: { + /* Make the fsm key in int format. */ + Key fsmKey = makeFsmKeyNum( token.data, token.loc, pd ); + /* Make the new machine. */ + rtnVal = new FsmAp(); + rtnVal->concatFsm( fsmKey ); + break; + } + case LitString: { + /* Make the array of keys in int format. */ + long length; + bool caseInsensitive; + char *data = prepareLitString( token.loc, token.data, token.length, + length, caseInsensitive ); + Key *arr = new Key[length]; + makeFsmKeyArray( arr, data, length, pd ); + + /* Make the new machine. */ + rtnVal = new FsmAp(); + if ( caseInsensitive ) + rtnVal->concatFsmCI( arr, length ); + else + rtnVal->concatFsm( arr, length ); + delete[] data; + delete[] arr; + break; + }} + return rtnVal; +} + +/* Clean up after a regular expression object. */ +RegExpr::~RegExpr() +{ + switch ( type ) { + case RecurseItem: + delete regExpr; + delete item; + break; + case Empty: + break; + } +} + +/* Evaluate a regular expression object. */ +FsmAp *RegExpr::walk( ParseData *pd, RegExpr *rootRegex ) +{ + /* This is the root regex, pass down a pointer to this. */ + if ( rootRegex == 0 ) + rootRegex = this; + + FsmAp *rtnVal = 0; + switch ( type ) { + case RecurseItem: { + /* Walk both items. */ + rtnVal = regExpr->walk( pd, rootRegex ); + FsmAp *fsm2 = item->walk( pd, rootRegex ); + rtnVal->concatOp( fsm2 ); + break; + } + case Empty: { + rtnVal = new FsmAp(); + rtnVal->lambdaFsm(); + break; + } + } + return rtnVal; +} + +/* Clean up after an item in a regular expression. */ +ReItem::~ReItem() +{ + switch ( type ) { + case Data: + case Dot: + break; + case OrBlock: + case NegOrBlock: + delete orBlock; + break; + } +} + +/* Evaluate a regular expression object. */ +FsmAp *ReItem::walk( ParseData *pd, RegExpr *rootRegex ) +{ + /* The fsm to return, is the alphabet signed? */ + FsmAp *rtnVal = 0; + + switch ( type ) { + case Data: { + /* Move the data into an integer array and make a concat fsm. */ + Key *arr = new Key[token.length]; + makeFsmKeyArray( arr, token.data, token.length, pd ); + + /* Make the concat fsm. */ + rtnVal = new FsmAp(); + if ( rootRegex != 0 && rootRegex->caseInsensitive ) + rtnVal->concatFsmCI( arr, token.length ); + else + rtnVal->concatFsm( arr, token.length ); + delete[] arr; + break; + } + case Dot: { + /* Make the dot fsm. */ + rtnVal = dotFsm( pd ); + break; + } + case OrBlock: { + /* Get the or block and minmize it. */ + rtnVal = orBlock->walk( pd, rootRegex ); + if ( rtnVal == 0 ) { + rtnVal = new FsmAp(); + rtnVal->lambdaFsm(); + } + rtnVal->minimizePartition2(); + break; + } + case NegOrBlock: { + /* Get the or block and minimize it. */ + FsmAp *fsm = orBlock->walk( pd, rootRegex ); + fsm->minimizePartition2(); + + /* Make a dot fsm and subtract from it. */ + rtnVal = dotFsm( pd ); + rtnVal->subtractOp( fsm ); + rtnVal->minimizePartition2(); + break; + } + } + + /* If the item is followed by a star, then apply the star op. */ + if ( star ) { + if ( rtnVal->startState->isFinState() ) { + warning(loc) << "applying kleene star to a machine that " + "accepts zero length word" << endl; + } + + rtnVal->starOp(); + rtnVal->minimizePartition2(); + } + return rtnVal; +} + +/* Clean up after an or block of a regular expression. */ +ReOrBlock::~ReOrBlock() +{ + switch ( type ) { + case RecurseItem: + delete orBlock; + delete item; + break; + case Empty: + break; + } +} + + +/* Evaluate an or block of a regular expression. */ +FsmAp *ReOrBlock::walk( ParseData *pd, RegExpr *rootRegex ) +{ + FsmAp *rtnVal = 0; + switch ( type ) { + case RecurseItem: { + /* Evaluate the two fsm. */ + FsmAp *fsm1 = orBlock->walk( pd, rootRegex ); + FsmAp *fsm2 = item->walk( pd, rootRegex ); + if ( fsm1 == 0 ) + rtnVal = fsm2; + else { + fsm1->unionOp( fsm2 ); + rtnVal = fsm1; + } + break; + } + case Empty: { + rtnVal = 0; + break; + } + } + return rtnVal;; +} + +/* Evaluate an or block item of a regular expression. */ +FsmAp *ReOrItem::walk( ParseData *pd, RegExpr *rootRegex ) +{ + /* The return value, is the alphabet signed? */ + FsmAp *rtnVal = 0; + switch ( type ) { + case Data: { + /* Make the or machine. */ + rtnVal = new FsmAp(); + + /* Put the or data into an array of ints. Note that we find unique + * keys. Duplicates are silently ignored. The alternative would be to + * issue warning or an error but since we can't with [a0-9a] or 'a' | + * 'a' don't bother here. */ + KeySet keySet; + makeFsmUniqueKeyArray( keySet, token.data, token.length, + rootRegex != 0 ? rootRegex->caseInsensitive : false, pd ); + + /* Run the or operator. */ + rtnVal->orFsm( keySet.data, keySet.length() ); + break; + } + case Range: { + /* Make the upper and lower keys. */ + Key lowKey = makeFsmKeyChar( lower, pd ); + Key highKey = makeFsmKeyChar( upper, pd ); + + /* Validate the range. */ + if ( lowKey > highKey ) { + /* Recover by setting upper to lower; */ + error(loc) << "lower end of range is greater then upper end" << endl; + highKey = lowKey; + } + + /* Make the range machine. */ + rtnVal = new FsmAp(); + rtnVal->rangeFsm( lowKey, highKey ); + + if ( rootRegex != 0 && rootRegex->caseInsensitive ) { + if ( lowKey <= 'Z' && 'A' <= highKey ) { + Key otherLow = lowKey < 'A' ? Key('A') : lowKey; + Key otherHigh = 'Z' < highKey ? Key('Z') : highKey; + + otherLow = 'a' + ( otherLow - 'A' ); + otherHigh = 'a' + ( otherHigh - 'A' ); + + FsmAp *otherRange = new FsmAp(); + otherRange->rangeFsm( otherLow, otherHigh ); + rtnVal->unionOp( otherRange ); + rtnVal->minimizePartition2(); + } + else if ( lowKey <= 'z' && 'a' <= highKey ) { + Key otherLow = lowKey < 'a' ? Key('a') : lowKey; + Key otherHigh = 'z' < highKey ? Key('z') : highKey; + + otherLow = 'A' + ( otherLow - 'a' ); + otherHigh = 'A' + ( otherHigh - 'a' ); + + FsmAp *otherRange = new FsmAp(); + otherRange->rangeFsm( otherLow, otherHigh ); + rtnVal->unionOp( otherRange ); + rtnVal->minimizePartition2(); + } + } + + break; + }} + return rtnVal; +} diff --git a/contrib/tools/ragel6/parsetree.h b/contrib/tools/ragel6/parsetree.h new file mode 100644 index 0000000000..954542e94d --- /dev/null +++ b/contrib/tools/ragel6/parsetree.h @@ -0,0 +1,772 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _PARSETREE_H +#define _PARSETREE_H + +#include "ragel.h" +#include "avlmap.h" +#include "bstmap.h" +#include "vector.h" +#include "dlist.h" + +struct NameInst; + +/* Types of builtin machines. */ +enum BuiltinMachine +{ + BT_Any, + BT_Ascii, + BT_Extend, + BT_Alpha, + BT_Digit, + BT_Alnum, + BT_Lower, + BT_Upper, + BT_Cntrl, + BT_Graph, + BT_Print, + BT_Punct, + BT_Space, + BT_Xdigit, + BT_Lambda, + BT_Empty +}; + + +struct ParseData; + +/* Leaf type. */ +struct Literal; + +/* Tree nodes. */ + +struct Term; +struct FactorWithAug; +struct FactorWithRep; +struct FactorWithNeg; +struct Factor; +struct Expression; +struct Join; +struct MachineDef; +struct LongestMatch; +struct LongestMatchPart; +struct LmPartList; +struct Range; +struct LengthDef; + +/* Type of augmentation. Describes locations in the machine. */ +enum AugType +{ + /* Transition actions/priorities. */ + at_start, + at_all, + at_finish, + at_leave, + + /* Global error actions. */ + at_start_gbl_error, + at_all_gbl_error, + at_final_gbl_error, + at_not_start_gbl_error, + at_not_final_gbl_error, + at_middle_gbl_error, + + /* Local error actions. */ + at_start_local_error, + at_all_local_error, + at_final_local_error, + at_not_start_local_error, + at_not_final_local_error, + at_middle_local_error, + + /* To State Action embedding. */ + at_start_to_state, + at_all_to_state, + at_final_to_state, + at_not_start_to_state, + at_not_final_to_state, + at_middle_to_state, + + /* From State Action embedding. */ + at_start_from_state, + at_all_from_state, + at_final_from_state, + at_not_start_from_state, + at_not_final_from_state, + at_middle_from_state, + + /* EOF Action embedding. */ + at_start_eof, + at_all_eof, + at_final_eof, + at_not_start_eof, + at_not_final_eof, + at_middle_eof +}; + +/* IMPORTANT: These must follow the same order as the state augs in AugType + * since we will be using this to compose AugType. */ +enum StateAugType +{ + sat_start = 0, + sat_all, + sat_final, + sat_not_start, + sat_not_final, + sat_middle +}; + +struct Action; +struct PriorDesc; +struct RegExpr; +struct ReItem; +struct ReOrBlock; +struct ReOrItem; +struct ExplicitMachine; +struct InlineItem; +struct InlineList; + +/* Reference to a named state. */ +typedef Vector<char*> NameRef; +typedef Vector<NameRef*> NameRefList; +typedef Vector<NameInst*> NameTargList; + +/* Structure for storing location of epsilon transitons. */ +struct EpsilonLink +{ + EpsilonLink( const InputLoc &loc, NameRef &target ) + : loc(loc), target(target) { } + + InputLoc loc; + NameRef target; +}; + +struct Label +{ + Label( const InputLoc &loc, char *data ) + : loc(loc), data(data) { } + + InputLoc loc; + char *data; +}; + +/* Structrue represents an action assigned to some FactorWithAug node. The + * factor with aug will keep an array of these. */ +struct ParserAction +{ + ParserAction( const InputLoc &loc, AugType type, int localErrKey, Action *action ) + : loc(loc), type(type), localErrKey(localErrKey), action(action) { } + + InputLoc loc; + AugType type; + int localErrKey; + Action *action; +}; + +struct ConditionTest +{ + ConditionTest( const InputLoc &loc, AugType type, Action *action, bool sense ) : + loc(loc), type(type), action(action), sense(sense) { } + + InputLoc loc; + AugType type; + Action *action; + bool sense; +}; + +struct Token +{ + char *data; + int length; + InputLoc loc; + + void append( const Token &other ); + void set( const char *str, int len ); +}; + +char *prepareLitString( const InputLoc &loc, const char *src, long length, + long &resLen, bool &caseInsensitive ); + +/* Store the value and type of a priority augmentation. */ +struct PriorityAug +{ + PriorityAug( AugType type, int priorKey, int priorValue ) : + type(type), priorKey(priorKey), priorValue(priorValue) { } + + AugType type; + int priorKey; + int priorValue; +}; + +/* + * A Variable Definition + */ +struct VarDef +{ + VarDef( const char *name, MachineDef *machineDef ) + : name(name), machineDef(machineDef), isExport(false) { } + + /* Parse tree traversal. */ + FsmAp *walk( ParseData *pd ); + void makeNameTree( const InputLoc &loc, ParseData *pd ); + void resolveNameRefs( ParseData *pd ); + + const char *name; + MachineDef *machineDef; + bool isExport; +}; + + +/* + * LongestMatch + * + * Wherever possible the item match will execute on the character. If not + * possible the item match will execute on a lookahead character and either + * hold the current char (if one away) or backup. + * + * How to handle the problem of backing up over a buffer break? + * + * Don't want to use pending out transitions for embedding item match because + * the role of item match action is different: it may sometimes match on the + * final transition, or may match on a lookahead character. + * + * Don't want to invent a new operator just for this. So just trail action + * after machine, this means we can only use literal actions. + * + * The item action may + * + * What states of the machine will be final. The item actions that wrap around + * on the last character will go straight to the start state. + * + * Some transitions will be lookahead transitions, they will hold the current + * character. Crossing them with regular transitions must be restricted + * because it does not make sense. The transition cannot simultaneously hold + * and consume the current character. + */ +struct LongestMatchPart +{ + LongestMatchPart( Join *join, Action *action, + InputLoc &semiLoc, int longestMatchId ) + : + join(join), action(action), semiLoc(semiLoc), + longestMatchId(longestMatchId), inLmSelect(false) { } + + InputLoc getLoc(); + + Join *join; + Action *action; + InputLoc semiLoc; + + Action *setActId; + Action *actOnLast; + Action *actOnNext; + Action *actLagBehind; + int longestMatchId; + bool inLmSelect; + LongestMatch *longestMatch; + + LongestMatchPart *prev, *next; +}; + +/* Declare a new type so that ptreetypes.h need not include dlist.h. */ +struct LmPartList : DList<LongestMatchPart> {}; + +struct LongestMatch +{ + /* Construct with a list of joins */ + LongestMatch( const InputLoc &loc, LmPartList *longestMatchList ) : + loc(loc), longestMatchList(longestMatchList), name(0), + lmSwitchHandlesError(false) { } + + /* Tree traversal. */ + FsmAp *walk( ParseData *pd ); + void makeNameTree( ParseData *pd ); + void resolveNameRefs( ParseData *pd ); + void transferScannerLeavingActions( FsmAp *graph ); + void runLongestMatch( ParseData *pd, FsmAp *graph ); + Action *newAction( ParseData *pd, const InputLoc &loc, const char *name, + InlineList *inlineList ); + void makeActions( ParseData *pd ); + void findName( ParseData *pd ); + void restart( FsmAp *graph, TransAp *trans ); + + InputLoc loc; + LmPartList *longestMatchList; + const char *name; + + Action *lmActSelect; + bool lmSwitchHandlesError; + + LongestMatch *next, *prev; +}; + + +/* List of Expressions. */ +typedef DList<Expression> ExprList; + +struct MachineDef +{ + enum Type { + JoinType, + LongestMatchType, + LengthDefType + }; + + MachineDef( Join *join ) + : join(join), longestMatch(0), lengthDef(0), type(JoinType) {} + MachineDef( LongestMatch *longestMatch ) + : join(0), longestMatch(longestMatch), lengthDef(0), type(LongestMatchType) {} + MachineDef( LengthDef *lengthDef ) + : join(0), longestMatch(0), lengthDef(lengthDef), type(LengthDefType) {} + + FsmAp *walk( ParseData *pd ); + void makeNameTree( ParseData *pd ); + void resolveNameRefs( ParseData *pd ); + + Join *join; + LongestMatch *longestMatch; + LengthDef *lengthDef; + Type type; +}; + +/* + * Join + */ +struct Join +{ + /* Construct with the first expression. */ + Join( Expression *expr ); + Join( const InputLoc &loc, Expression *expr ); + + /* Tree traversal. */ + FsmAp *walk( ParseData *pd ); + FsmAp *walkJoin( ParseData *pd ); + void makeNameTree( ParseData *pd ); + void resolveNameRefs( ParseData *pd ); + + /* Data. */ + InputLoc loc; + ExprList exprList; +}; + +/* + * Expression + */ +struct Expression +{ + enum Type { + OrType, + IntersectType, + SubtractType, + StrongSubtractType, + TermType, + BuiltinType + }; + + /* Construct with an expression on the left and a term on the right. */ + Expression( Expression *expression, Term *term, Type type ) : + expression(expression), term(term), + type(type), prev(this), next(this) { } + + /* Construct with only a term. */ + Expression( Term *term ) : + expression(0), term(term), + type(TermType) , prev(this), next(this) { } + + /* Construct with a builtin type. */ + Expression( BuiltinMachine builtin ) : + expression(0), term(0), builtin(builtin), + type(BuiltinType), prev(this), next(this) { } + + ~Expression(); + + /* Tree traversal. */ + FsmAp *walk( ParseData *pd, bool lastInSeq = true ); + void makeNameTree( ParseData *pd ); + void resolveNameRefs( ParseData *pd ); + + /* Node data. */ + Expression *expression; + Term *term; + BuiltinMachine builtin; + Type type; + + Expression *prev, *next; +}; + +/* + * Term + */ +struct Term +{ + enum Type { + ConcatType, + RightStartType, + RightFinishType, + LeftType, + FactorWithAugType + }; + + Term( Term *term, FactorWithAug *factorWithAug ) : + term(term), factorWithAug(factorWithAug), type(ConcatType) { } + + Term( Term *term, FactorWithAug *factorWithAug, Type type ) : + term(term), factorWithAug(factorWithAug), type(type) { } + + Term( FactorWithAug *factorWithAug ) : + term(0), factorWithAug(factorWithAug), type(FactorWithAugType) { } + + ~Term(); + + FsmAp *walk( ParseData *pd, bool lastInSeq = true ); + void makeNameTree( ParseData *pd ); + void resolveNameRefs( ParseData *pd ); + + Term *term; + FactorWithAug *factorWithAug; + Type type; + + /* Priority descriptor for RightFinish type. */ + PriorDesc priorDescs[2]; +}; + + +/* Third level of precedence. Augmenting nodes with actions and priorities. */ +struct FactorWithAug +{ + FactorWithAug( FactorWithRep *factorWithRep ) : + priorDescs(0), factorWithRep(factorWithRep) { } + ~FactorWithAug(); + + /* Tree traversal. */ + FsmAp *walk( ParseData *pd ); + void makeNameTree( ParseData *pd ); + void resolveNameRefs( ParseData *pd ); + + void assignActions( ParseData *pd, FsmAp *graph, int *actionOrd ); + void assignPriorities( FsmAp *graph, int *priorOrd ); + + void assignConditions( FsmAp *graph ); + + /* Actions and priorities assigned to the factor node. */ + Vector<ParserAction> actions; + Vector<PriorityAug> priorityAugs; + PriorDesc *priorDescs; + Vector<Label> labels; + Vector<EpsilonLink> epsilonLinks; + Vector<ConditionTest> conditions; + + FactorWithRep *factorWithRep; +}; + +/* Fourth level of precedence. Trailing unary operators. Provide kleen star, + * optional and plus. */ +struct FactorWithRep +{ + enum Type { + StarType, + StarStarType, + OptionalType, + PlusType, + ExactType, + MaxType, + MinType, + RangeType, + FactorWithNegType + }; + + FactorWithRep( const InputLoc &loc, FactorWithRep *factorWithRep, + int lowerRep, int upperRep, Type type ) : + loc(loc), factorWithRep(factorWithRep), + factorWithNeg(0), lowerRep(lowerRep), + upperRep(upperRep), type(type) { } + + FactorWithRep( FactorWithNeg *factorWithNeg ) + : factorWithNeg(factorWithNeg), type(FactorWithNegType) { } + + ~FactorWithRep(); + + /* Tree traversal. */ + FsmAp *walk( ParseData *pd ); + void makeNameTree( ParseData *pd ); + void resolveNameRefs( ParseData *pd ); + + InputLoc loc; + FactorWithRep *factorWithRep; + FactorWithNeg *factorWithNeg; + int lowerRep, upperRep; + Type type; + + /* Priority descriptor for StarStar type. */ + PriorDesc priorDescs[2]; +}; + +/* Fifth level of precedence. Provides Negation. */ +struct FactorWithNeg +{ + enum Type { + NegateType, + CharNegateType, + FactorType + }; + + FactorWithNeg( const InputLoc &loc, FactorWithNeg *factorWithNeg, Type type) : + loc(loc), factorWithNeg(factorWithNeg), factor(0), type(type) { } + + FactorWithNeg( Factor *factor ) : + factorWithNeg(0), factor(factor), type(FactorType) { } + + ~FactorWithNeg(); + + /* Tree traversal. */ + FsmAp *walk( ParseData *pd ); + void makeNameTree( ParseData *pd ); + void resolveNameRefs( ParseData *pd ); + + InputLoc loc; + FactorWithNeg *factorWithNeg; + Factor *factor; + Type type; +}; + +/* + * Factor + */ +struct Factor +{ + /* Language elements a factor node can be. */ + enum Type { + LiteralType, + RangeType, + OrExprType, + RegExprType, + ReferenceType, + ParenType, + LongestMatchType, + }; + + /* Construct with a literal fsm. */ + Factor( Literal *literal ) : + literal(literal), type(LiteralType) { } + + /* Construct with a range. */ + Factor( Range *range ) : + range(range), type(RangeType) { } + + /* Construct with the or part of a regular expression. */ + Factor( ReItem *reItem ) : + reItem(reItem), type(OrExprType) { } + + /* Construct with a regular expression. */ + Factor( RegExpr *regExpr ) : + regExpr(regExpr), type(RegExprType) { } + + /* Construct with a reference to a var def. */ + Factor( const InputLoc &loc, VarDef *varDef ) : + loc(loc), varDef(varDef), type(ReferenceType) {} + + /* Construct with a parenthesized join. */ + Factor( Join *join ) : + join(join), type(ParenType) {} + + /* Construct with a longest match operator. */ + Factor( LongestMatch *longestMatch ) : + longestMatch(longestMatch), type(LongestMatchType) {} + + /* Cleanup. */ + ~Factor(); + + /* Tree traversal. */ + FsmAp *walk( ParseData *pd ); + void makeNameTree( ParseData *pd ); + void resolveNameRefs( ParseData *pd ); + + InputLoc loc; + Literal *literal; + Range *range; + ReItem *reItem; + RegExpr *regExpr; + VarDef *varDef; + Join *join; + LongestMatch *longestMatch; + int lower, upper; + Type type; +}; + +/* A range machine. Only ever composed of two literals. */ +struct Range +{ + Range( Literal *lowerLit, Literal *upperLit ) + : lowerLit(lowerLit), upperLit(upperLit) { } + + ~Range(); + FsmAp *walk( ParseData *pd ); + + Literal *lowerLit; + Literal *upperLit; +}; + +/* Some literal machine. Can be a number or literal string. */ +struct Literal +{ + enum LiteralType { Number, LitString }; + + Literal( const Token &token, LiteralType type ) + : token(token), type(type) { } + + FsmAp *walk( ParseData *pd ); + + Token token; + LiteralType type; +}; + +/* Regular expression. */ +struct RegExpr +{ + enum RegExpType { RecurseItem, Empty }; + + /* Constructors. */ + RegExpr() : + type(Empty), caseInsensitive(false) { } + RegExpr(RegExpr *regExpr, ReItem *item) : + regExpr(regExpr), item(item), + type(RecurseItem), caseInsensitive(false) { } + + ~RegExpr(); + FsmAp *walk( ParseData *pd, RegExpr *rootRegex ); + + RegExpr *regExpr; + ReItem *item; + RegExpType type; + bool caseInsensitive; +}; + +/* An item in a regular expression. */ +struct ReItem +{ + enum ReItemType { Data, Dot, OrBlock, NegOrBlock }; + + ReItem( const InputLoc &loc, const Token &token ) + : loc(loc), token(token), star(false), type(Data) { } + ReItem( const InputLoc &loc, ReItemType type ) + : loc(loc), star(false), type(type) { } + ReItem( const InputLoc &loc, ReOrBlock *orBlock, ReItemType type ) + : loc(loc), orBlock(orBlock), star(false), type(type) { } + + ~ReItem(); + FsmAp *walk( ParseData *pd, RegExpr *rootRegex ); + + InputLoc loc; + Token token; + ReOrBlock *orBlock; + bool star; + ReItemType type; +}; + +/* An or block item. */ +struct ReOrBlock +{ + enum ReOrBlockType { RecurseItem, Empty }; + + /* Constructors. */ + ReOrBlock() + : type(Empty) { } + ReOrBlock(ReOrBlock *orBlock, ReOrItem *item) + : orBlock(orBlock), item(item), type(RecurseItem) { } + + ~ReOrBlock(); + FsmAp *walk( ParseData *pd, RegExpr *rootRegex ); + + ReOrBlock *orBlock; + ReOrItem *item; + ReOrBlockType type; +}; + +/* An item in an or block. */ +struct ReOrItem +{ + enum ReOrItemType { Data, Range }; + + ReOrItem( const InputLoc &loc, const Token &token ) + : loc(loc), token(token), type(Data) {} + ReOrItem( const InputLoc &loc, char lower, char upper ) + : loc(loc), lower(lower), upper(upper), type(Range) { } + + FsmAp *walk( ParseData *pd, RegExpr *rootRegex ); + + InputLoc loc; + Token token; + char lower; + char upper; + ReOrItemType type; +}; + + +/* + * Inline code tree + */ +struct InlineList; +struct InlineItem +{ + enum Type + { + Text, Goto, Call, Next, GotoExpr, CallExpr, NextExpr, Ret, PChar, + Char, Hold, Curs, Targs, Entry, Exec, LmSwitch, LmSetActId, + LmSetTokEnd, LmOnLast, LmOnNext, LmOnLagBehind, LmInitAct, + LmInitTokStart, LmSetTokStart, Break + }; + + InlineItem( const InputLoc &loc, char *data, Type type ) : + loc(loc), data(data), nameRef(0), children(0), type(type) { } + + InlineItem( const InputLoc &loc, NameRef *nameRef, Type type ) : + loc(loc), data(0), nameRef(nameRef), children(0), type(type) { } + + InlineItem( const InputLoc &loc, LongestMatch *longestMatch, + LongestMatchPart *longestMatchPart, Type type ) : loc(loc), data(0), + nameRef(0), children(0), longestMatch(longestMatch), + longestMatchPart(longestMatchPart), type(type) { } + + InlineItem( const InputLoc &loc, NameInst *nameTarg, Type type ) : + loc(loc), data(0), nameRef(0), nameTarg(nameTarg), children(0), + type(type) { } + + InlineItem( const InputLoc &loc, Type type ) : + loc(loc), data(0), nameRef(0), children(0), type(type) { } + + InputLoc loc; + char *data; + NameRef *nameRef; + NameInst *nameTarg; + InlineList *children; + LongestMatch *longestMatch; + LongestMatchPart *longestMatchPart; + Type type; + + InlineItem *prev, *next; +}; + +/* Normally this would be atypedef, but that would entail including DList from + * ptreetypes, which should be just typedef forwards. */ +struct InlineList : public DList<InlineItem> { }; + +#endif diff --git a/contrib/tools/ragel6/pcheck.h b/contrib/tools/ragel6/pcheck.h new file mode 100644 index 0000000000..0e388f8ab4 --- /dev/null +++ b/contrib/tools/ragel6/pcheck.h @@ -0,0 +1,48 @@ +/* + * Copyright 2001, 2002 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _PCHECK_H +#define _PCHECK_H + +class ParamCheck +{ +public: + ParamCheck( const char *paramSpec, int argc, const char **argv); + + bool check(); + + const char *paramArg; /* The argument to the parameter. */ + char parameter; /* The parameter matched. */ + enum { match, invalid, noparam } state; + + const char *argOffset; /* If we are reading params inside an + * arg this points to the offset. */ + + const char *curArg; /* Pointer to the current arg. */ + int iCurArg; /* Index to the current arg. */ + +private: + const char *paramSpec; /* Parameter spec supplied by the coder. */ + int argc; /* Arguement data from the command line. */ + const char **argv; +}; + +#endif diff --git a/contrib/tools/ragel6/ragel.h b/contrib/tools/ragel6/ragel.h new file mode 100644 index 0000000000..2f4d6ffb60 --- /dev/null +++ b/contrib/tools/ragel6/ragel.h @@ -0,0 +1,120 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RAGEL_H +#define _RAGEL_H + +#include <stdio.h> +#include <iostream> +#include <fstream> +#include <string> +#include "vector.h" +#include "config.h" +#include "common.h" + +#define PROGNAME "ragel" + +/* Target output style. */ +enum CodeStyle +{ + GenTables, + GenFTables, + GenFlat, + GenFFlat, + GenGoto, + GenFGoto, + GenIpGoto, + GenSplit +}; + +/* To what degree are machine minimized. */ +enum MinimizeLevel { + MinimizeApprox, + MinimizeStable, + MinimizePartition1, + MinimizePartition2 +}; + +enum MinimizeOpt { + MinimizeNone, + MinimizeEnd, + MinimizeMostOps, + MinimizeEveryOp +}; + +/* Target implementation */ +enum RubyImplEnum +{ + MRI, + Rubinius +}; + +/* Options. */ +extern MinimizeLevel minimizeLevel; +extern MinimizeOpt minimizeOpt; +extern const char *machineSpec, *machineName; +extern bool printStatistics; +extern bool wantDupsRemoved; +extern bool generateDot; +extern bool generateXML; +extern RubyImplEnum rubyImpl; + +/* Error reporting format. */ +enum ErrorFormat { + ErrorFormatGNU, + ErrorFormatMSVC, +}; + +extern ErrorFormat errorFormat; +extern int gblErrorCount; +extern char mainMachine[]; + +InputLoc makeInputLoc( const char *fileName, int line = 0, int col = 0 ); +std::ostream &operator<<( std::ostream &out, const InputLoc &loc ); + +/* Error reporting. */ +std::ostream &error(); +std::ostream &error( const InputLoc &loc ); +std::ostream &warning( const InputLoc &loc ); + +struct XmlParser; + +void xmlEscapeHost( std::ostream &out, char *data, long len ); + +extern CodeStyle codeStyle; + +/* IO filenames and stream. */ +extern bool displayPrintables; +extern int gblErrorCount; + +/* Options. */ +extern int numSplitPartitions; +extern bool noLineDirectives; + +std::ostream &error(); + +/* Target language and output style. */ +extern CodeStyle codeStyle; + +extern int numSplitPartitions; +extern bool noLineDirectives; + +#endif diff --git a/contrib/tools/ragel6/rbxgoto.cpp b/contrib/tools/ragel6/rbxgoto.cpp new file mode 100644 index 0000000000..932fdf35f2 --- /dev/null +++ b/contrib/tools/ragel6/rbxgoto.cpp @@ -0,0 +1,831 @@ +/* + * Copyright 2007 Victor Hugo Borja <vic@rubyforge.org> + * 2006-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <stdio.h> +#include <string> + +#include "rbxgoto.h" +#include "ragel.h" +#include "redfsm.h" +#include "bstmap.h" +#include "gendata.h" + +using std::ostream; +using std::string; + +inline string label(string a, int i) +{ + return a + itoa(i); +} + +ostream &RbxGotoCodeGen::rbxLabel(ostream &out, string label) +{ + out << "Rubinius.asm { @labels[:_" << FSM_NAME() << "_" << label << "].set! }\n"; + return out; +} + +ostream &RbxGotoCodeGen::rbxGoto(ostream &out, string label) +{ + out << "Rubinius.asm { goto @labels[:_" << FSM_NAME() << "_" << label << "] }\n"; + return out; +} + +/* Emit the goto to take for a given transition. */ +std::ostream &RbxGotoCodeGen::TRANS_GOTO( RedTransAp *trans, int level ) +{ + out << TABS(level); + return rbxGoto(out, label("tr",trans->id)); +} + +std::ostream &RbxGotoCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RbxGotoCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RbxGotoCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, true ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RbxGotoCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +void RbxGotoCodeGen::GOTO_HEADER( RedStateAp *state ) +{ + /* Label the state. */ + out << "when " << state->id << " then\n"; +} + + +void RbxGotoCodeGen::emitSingleSwitch( RedStateAp *state ) +{ + /* Load up the singles. */ + int numSingles = state->outSingle.length(); + RedTransEl *data = state->outSingle.data; + + if ( numSingles == 1 ) { + /* If there is a single single key then write it out as an if. */ + out << "\tif " << GET_WIDE_KEY(state) << " == " << + KEY(data[0].lowKey) << " \n\t\t"; + + /* Virtual function for writing the target of the transition. */ + TRANS_GOTO(data[0].value, 0) << "\n"; + + out << "end\n"; + } + else if ( numSingles > 1 ) { + /* Write out single keys in a switch if there is more than one. */ + out << "\tcase " << GET_WIDE_KEY(state) << "\n"; + + /* Write out the single indicies. */ + for ( int j = 0; j < numSingles; j++ ) { + out << "\t\twhen " << KEY(data[j].lowKey) << " then\n"; + TRANS_GOTO(data[j].value, 0) << "\n"; + } + + /* Close off the transition switch. */ + out << "\tend\n"; + } +} + +void RbxGotoCodeGen::emitRangeBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + RedTransEl *data = state->outRange.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid].lowKey == keyOps->minKey; + bool limitHigh = data[mid].highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " \n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "elsif " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " \n"; + emitRangeBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + out << TABS(level) << "end\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " < " << + KEY(data[mid].lowKey) << " then\n"; + emitRangeBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "elsif" << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " )\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + out << TABS(level) << "end\n"; + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " > " << + KEY(data[mid].highKey) << " \n"; + emitRangeBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "else\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + else { + out << TABS(level) << "elsif " << GET_WIDE_KEY(state) << " >= " << + KEY(data[mid].lowKey) << " then\n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + out << TABS(level) << "end\n"; + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " && " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " \n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + out << TABS(level) << "end\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if " << GET_WIDE_KEY(state) << " <= " << + KEY(data[mid].highKey) << " \n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + out << TABS(level) << "end\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid].lowKey) << " <= " << + GET_WIDE_KEY(state) << " \n"; + TRANS_GOTO(data[mid].value, level+1) << "\n"; + out << TABS(level) << "end\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + TRANS_GOTO(data[mid].value, level+1) << "\n"; + } + } +} + +void RbxGotoCodeGen::STATE_GOTO_ERROR() +{ + /* Label the state and bail immediately. */ + outLabelUsed = true; + RedStateAp *state = redFsm->errState; + out << "when " << state->id << " then\n"; + rbxGoto(out << " ", "_out") << "\n"; +} + +void RbxGotoCodeGen::COND_TRANSLATE( GenStateCond *stateCond, int level ) +{ + GenCondSpace *condSpace = stateCond->condSpace; + out << TABS(level) << "_widec = " << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << ");\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(level) << "if "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << "\n _widec += " << condValOffset << ";\n end"; + } +} + +void RbxGotoCodeGen::emitCondBSearch( RedStateAp *state, int level, int low, int high ) +{ + /* Get the mid position, staying on the lower end of the range. */ + int mid = (low + high) >> 1; + GenStateCond **data = state->stateCondVect.data; + + /* Determine if we need to look higher or lower. */ + bool anyLower = mid > low; + bool anyHigher = mid < high; + + /* Determine if the keys at mid are the limits of the alphabet. */ + bool limitLow = data[mid]->lowKey == keyOps->minKey; + bool limitHigh = data[mid]->highKey == keyOps->maxKey; + + if ( anyLower && anyHigher ) { + /* Can go lower and higher than mid. */ + out << TABS(level) << "if " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " \n"; + emitCondBSearch( state, level+1, low, mid-1 ); + out << TABS(level) << "elsif " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " \n"; + emitCondBSearch( state, level+1, mid+1, high ); + out << TABS(level) << "else\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "end\n"; + } + else if ( anyLower && !anyHigher ) { + /* Can go lower than mid but not higher. */ + out << TABS(level) << "if " << GET_KEY() << " < " << + KEY(data[mid]->lowKey) << " \n"; + emitCondBSearch( state, level+1, low, mid-1 ); + + /* if the higher is the highest in the alphabet then there is no + * sense testing it. */ + if ( limitHigh ) { + out << TABS(level) << "else\n"; + COND_TRANSLATE(data[mid], level+1); + } + else { + out << TABS(level) << "elsif " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " then\n"; + COND_TRANSLATE(data[mid], level+1); + } + out << TABS(level) << "end\n"; + + } + else if ( !anyLower && anyHigher ) { + /* Can go higher than mid but not lower. */ + out << TABS(level) << "if " << GET_KEY() << " > " << + KEY(data[mid]->highKey) << " \n"; + emitCondBSearch( state, level+1, mid+1, high ); + + /* If the lower end is the lowest in the alphabet then there is no + * sense testing it. */ + if ( limitLow ) { + out << TABS(level) << "else\n"; + COND_TRANSLATE(data[mid], level+1); + } + else { + out << TABS(level) << "elsif " << GET_KEY() << " >= " << + KEY(data[mid]->lowKey) << " then\n"; + COND_TRANSLATE(data[mid], level+1); + } + out << TABS(level) << "end\n"; + } + else { + /* Cannot go higher or lower than mid. It's mid or bust. What + * tests to do depends on limits of alphabet. */ + if ( !limitLow && !limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " && " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " then\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "end\n"; + } + else if ( limitLow && !limitHigh ) { + out << TABS(level) << "if " << GET_KEY() << " <= " << + KEY(data[mid]->highKey) << " then\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "end\n"; + } + else if ( !limitLow && limitHigh ) { + out << TABS(level) << "if " << KEY(data[mid]->lowKey) << " <= " << + GET_KEY() << " then\n"; + COND_TRANSLATE(data[mid], level+1); + out << TABS(level) << "end\n"; + } + else { + /* Both high and low are at the limit. No tests to do. */ + COND_TRANSLATE(data[mid], level); + } + } +} + +std::ostream &RbxGotoCodeGen::STATE_GOTOS() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st == redFsm->errState ) + STATE_GOTO_ERROR(); + else { + /* Writing code above state gotos. */ + GOTO_HEADER( st ); + + if ( st->stateCondVect.length() > 0 ) { + out << " _widec = " << GET_KEY() << ";\n"; + emitCondBSearch( st, 1, 0, st->stateCondVect.length() - 1 ); + } + + /* Try singles. */ + if ( st->outSingle.length() > 0 ) + emitSingleSwitch( st ); + + /* Default case is to binary search for the ranges, if that fails then */ + if ( st->outRange.length() > 0 ) + emitRangeBSearch( st, 1, 0, st->outRange.length() - 1 ); + + /* Write the default transition. */ + TRANS_GOTO( st->defTrans, 1 ) << "\n"; + } + } + return out; +} + +std::ostream &RbxGotoCodeGen::TRANSITIONS() +{ + /* Emit any transitions that have functions and that go to + * this state. */ + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) { + /* Write the label for the transition so it can be jumped to. */ + rbxLabel(out << " ", label("tr", trans->id)) << "\n"; + + /* Destination state. */ + if ( trans->action != 0 && trans->action->anyCurStateRef() ) + out << "_ps = " << vCS() << "'n"; + out << vCS() << " = " << trans->targ->id << "\n"; + + if ( trans->action != 0 ) { + /* Write out the transition func. */ + rbxGoto(out, label("f", trans->action->actListId)) << "\n"; + } + else { + /* No code to execute, just loop around. */ + rbxGoto(out, "_again") << "\n"; + } + } + return out; +} + +std::ostream &RbxGotoCodeGen::EXEC_FUNCS() +{ + /* Make labels that set acts and jump to execFuncs. Loop func indicies. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + rbxLabel(out, label("f", redAct->actListId)) << "\n" << + "_acts = " << itoa( redAct->location+1 ) << "\n"; + rbxGoto(out, "execFuncs") << "\n"; + } + } + + rbxLabel(out, "execFuncs") << + "\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while ( _nacts > 0 ) \n" + " _nacts -= 1\n" + " _acts += 1\n" + " case ( "<< A() << "[_acts-1] ) \n"; + ACTION_SWITCH(); + out << + " end\n" + " end \n"; + rbxGoto(out, "_again"); + return out; +} + +int RbxGotoCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +int RbxGotoCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +int RbxGotoCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + +std::ostream &RbxGotoCodeGen::TO_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = TO_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &RbxGotoCodeGen::FROM_STATE_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = FROM_STATE_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &RbxGotoCodeGen::EOF_ACTIONS() +{ + /* Take one off for the psuedo start state. */ + int numStates = redFsm->stateList.length(); + unsigned int *vals = new unsigned int[numStates]; + memset( vals, 0, sizeof(unsigned int)*numStates ); + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) + vals[st->id] = EOF_ACTION(st); + + out << "\t"; + for ( int st = 0; st < redFsm->nextStateId; st++ ) { + /* Write any eof action. */ + out << vals[st]; + if ( st < numStates-1 ) { + out << ", "; + if ( (st+1) % IALL == 0 ) + out << "\n\t"; + } + } + out << "\n"; + delete[] vals; + return out; +} + +std::ostream &RbxGotoCodeGen::FINISH_CASES() +{ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* States that are final and have an out action need a case. */ + if ( st->eofAction != 0 ) { + /* Write the case label. */ + out << "\t\twhen " << st->id << " then\n"; + + /* Write the goto func. */ + rbxGoto(out, label("f", st->eofAction->actListId)) << "\n"; + } + } + + return out; +} + +void RbxGotoCodeGen::GOTO( ostream &ret, int gotoDest, bool inFinish ) +{ + ret << "begin\n" << vCS() << " = " << gotoDest << " "; + rbxGoto(ret, "_again") << + "\nend\n"; +} + +void RbxGotoCodeGen::GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << "begin\n" << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ")"; + rbxGoto(ret, "_again") << + "\nend\n"; +} + +void RbxGotoCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void RbxGotoCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void RbxGotoCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void RbxGotoCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + +void RbxGotoCodeGen::CALL( ostream &ret, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "begin\n" + << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = " << + callDest << "; "; + rbxGoto(ret, "_again") << + "\nend\n"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void RbxGotoCodeGen::CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, prePushExpr, 0, false ); + } + + ret << "begin\n" << STACK() << "[" << TOP() << "++] = " << vCS() << "; " << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, targState, inFinish ); + ret << "); "; + rbxGoto(ret, "_again") << + "\nend\n"; + + if ( prePushExpr != 0 ) + ret << "}"; +} + +void RbxGotoCodeGen::RET( ostream &ret, bool inFinish ) +{ + ret << "begin\n" << vCS() << " = " << STACK() << "[--" << TOP() << "]; " ; + + if ( postPopExpr != 0 ) { + ret << "{"; + INLINE_LIST( ret, postPopExpr, 0, false ); + ret << "}"; + } + + rbxGoto(ret, "_again") << + "\nend\n"; +} + +void RbxGotoCodeGen::BREAK( ostream &ret, int targState ) +{ + outLabelUsed = true; + + out << + " begin\n" + " " << P() << " += 1\n" + " "; rbxGoto(ret, "_out") << "\n" + " end\n"; +} + +void RbxGotoCodeGen::writeData() +{ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void RbxGotoCodeGen::writeExec() +{ + outLabelUsed = false; + + out << " begin\n"; + + out << " Rubinius.asm { @labels = Hash.new { |h,k| h[k] = new_label } }\n"; + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = 0;\n"; + + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + { + out << " _acts, _nacts = nil\n"; + } + + if ( redFsm->anyConditions() ) + out << " _widec = nil\n"; + + out << "\n"; + + if ( !noEnd ) { + outLabelUsed = true; + out << + " if ( " << P() << " == " << PE() << " )\n"; + rbxGoto(out << " ", "_out") << "\n" << + " end\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n"; + rbxGoto(out << " ", "_out") << "\n" << + " end\n"; + } + + rbxLabel(out, "_resume") << "\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + + " _acts = " << ARR_OFF( A(), FSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + FROM_STATE_ACTION_SWITCH(); + out << + " }\n" + " }\n" + "\n"; + } + + out << + " case ( " << vCS() << " )\n"; + STATE_GOTOS(); + out << + " end # case\n" + "\n"; + TRANSITIONS() << + "\n"; + + if ( redFsm->anyRegActions() ) + EXEC_FUNCS() << "\n"; + + + rbxLabel(out, "_again") << "\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << ARR_OFF( A(), TSA() + "[" + vCS() + "]" ) << ";\n" + " _nacts = " << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + TO_STATE_ACTION_SWITCH(); + out << + " }\n" + " }\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + outLabelUsed = true; + out << + " if ( " << vCS() << " == " << redFsm->errState->id << " )\n"; + rbxGoto(out << " ", "_out") << "\n" << + " end" << "\n"; + } + + if ( !noEnd ) { + out << " " << P() << " += 1\n" + " if ( " << P() << " != " << PE() << " )\n"; + rbxGoto(out << " ", "_resume") << "\n" << + " end" << "\n"; + } + else { + out << + " " << P() << " += 1;\n"; + rbxGoto(out << " ", "_resume") << "\n"; + } + + if ( outLabelUsed ) + rbxLabel(out, "_out") << "\n"; + + out << " end\n"; +} + +void RbxGotoCodeGen::writeEOF() +{ + if ( redFsm->anyEofActions() ) { + out << + " {\n" + " _acts = " << + ARR_OFF( A(), EA() + "[" + vCS() + "]" ) << ";\n" + " " << " _nacts = " << " *_acts++;\n" + " while ( _nacts-- > 0 ) {\n" + " switch ( *_acts++ ) {\n"; + EOF_ACTION_SWITCH(); + out << + " }\n" + " }\n" + " }\n" + "\n"; + } +} + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/contrib/tools/ragel6/rbxgoto.h b/contrib/tools/ragel6/rbxgoto.h new file mode 100644 index 0000000000..291b656004 --- /dev/null +++ b/contrib/tools/ragel6/rbxgoto.h @@ -0,0 +1,97 @@ +/* + * Copyright 2007 Victor Hugo Borja <vic@rubyforge.org> + * 2006-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RBX_GOTOCODEGEN_H +#define _RBX_GOTOCODEGEN_H + +#include <iostream> +#include <string> +#include "rubycodegen.h" + +using std::string; + +class RbxGotoCodeGen : public RubyCodeGen +{ +public: + RbxGotoCodeGen( ostream &out ) : RubyCodeGen(out) {} + virtual ~RbxGotoCodeGen() {} + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + std::ostream &STATE_GOTOS(); + std::ostream &TRANSITIONS(); + std::ostream &EXEC_FUNCS(); + std::ostream &FINISH_CASES(); + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + int TO_STATE_ACTION( RedStateAp *state ); + int FROM_STATE_ACTION( RedStateAp *state ); + int EOF_ACTION( RedStateAp *state ); + + void COND_TRANSLATE( GenStateCond *stateCond, int level ); + void emitCondBSearch( RedStateAp *state, int level, int low, int high ); + void STATE_CONDS( RedStateAp *state, bool genDefault ); + + virtual std::ostream &TRANS_GOTO( RedTransAp *trans, int level ); + + void emitSingleSwitch( RedStateAp *state ); + void emitRangeBSearch( RedStateAp *state, int level, int low, int high ); + + /* Called from STATE_GOTOS just before writing the gotos */ + virtual void GOTO_HEADER( RedStateAp *state ); + virtual void STATE_GOTO_ERROR(); + + virtual void writeData(); + virtual void writeEOF(); + virtual void writeExec(); + + + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + +private: + ostream &rbxGoto(ostream &out, string label); + ostream &rbxLabel(ostream &out, string label); +}; + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ + +#endif diff --git a/contrib/tools/ragel6/redfsm.cpp b/contrib/tools/ragel6/redfsm.cpp new file mode 100644 index 0000000000..5b02d996c1 --- /dev/null +++ b/contrib/tools/ragel6/redfsm.cpp @@ -0,0 +1,584 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "redfsm.h" +#include "avlmap.h" +#include "mergesort.h" +#include <iostream> +#include <sstream> + +using std::ostringstream; + +string GenAction::nameOrLoc() +{ + if ( name != 0 ) + return string(name); + else { + ostringstream ret; + ret << loc.line << ":" << loc.col; + return ret.str(); + } +} + +RedFsmAp::RedFsmAp() +: + forcedErrorState(false), + nextActionId(0), + nextTransId(0), + startState(0), + errState(0), + errTrans(0), + firstFinState(0), + numFinStates(0), + bAnyToStateActions(false), + bAnyFromStateActions(false), + bAnyRegActions(false), + bAnyEofActions(false), + bAnyEofTrans(false), + bAnyActionGotos(false), + bAnyActionCalls(false), + bAnyActionRets(false), + bAnyActionByValControl(false), + bAnyRegActionRets(false), + bAnyRegActionByValControl(false), + bAnyRegNextStmt(false), + bAnyRegCurStateRef(false), + bAnyRegBreak(false), + bAnyConditions(false) +{ +} + +/* Does the machine have any actions. */ +bool RedFsmAp::anyActions() +{ + return actionMap.length() > 0; +} + +void RedFsmAp::depthFirstOrdering( RedStateAp *state ) +{ + /* Nothing to do if the state is already on the list. */ + if ( state->onStateList ) + return; + + /* Doing depth first, put state on the list. */ + state->onStateList = true; + stateList.append( state ); + + /* At this point transitions should only be in ranges. */ + assert( state->outSingle.length() == 0 ); + assert( state->defTrans == 0 ); + + /* Recurse on everything ranges. */ + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) { + if ( rtel->value->targ != 0 ) + depthFirstOrdering( rtel->value->targ ); + } +} + +/* Ordering states by transition connections. */ +void RedFsmAp::depthFirstOrdering() +{ + /* Init on state list flags. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) + st->onStateList = false; + + /* Clear out the state list, we will rebuild it. */ + int stateListLen = stateList.length(); + stateList.abandon(); + + /* Add back to the state list from the start state and all other entry + * points. */ + if ( startState != 0 ) + depthFirstOrdering( startState ); + for ( RedStateSet::Iter en = entryPoints; en.lte(); en++ ) + depthFirstOrdering( *en ); + if ( forcedErrorState ) + depthFirstOrdering( errState ); + + /* Make sure we put everything back on. */ + assert( stateListLen == stateList.length() ); +} + +/* Assign state ids by appearance in the state list. */ +void RedFsmAp::sequentialStateIds() +{ + /* Table based machines depend on the state numbers starting at zero. */ + nextStateId = 0; + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) + st->id = nextStateId++; +} + +/* Stable sort the states by final state status. */ +void RedFsmAp::sortStatesByFinal() +{ + /* Move forward through the list and throw final states onto the end. */ + RedStateAp *state = 0; + RedStateAp *next = stateList.head; + RedStateAp *last = stateList.tail; + while ( state != last ) { + /* Move forward and load up the next. */ + state = next; + next = state->next; + + /* Throw to the end? */ + if ( state->isFinal ) { + stateList.detach( state ); + stateList.append( state ); + } + } +} + +/* Assign state ids by final state state status. */ +void RedFsmAp::sortStateIdsByFinal() +{ + /* Table based machines depend on this starting at zero. */ + nextStateId = 0; + + /* First pass to assign non final ids. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + if ( ! st->isFinal ) + st->id = nextStateId++; + } + + /* Second pass to assign final ids. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + if ( st->isFinal ) + st->id = nextStateId++; + } +} + +struct CmpStateById +{ + static int compare( RedStateAp *st1, RedStateAp *st2 ) + { + if ( st1->id < st2->id ) + return -1; + else if ( st1->id > st2->id ) + return 1; + else + return 0; + } +}; + +void RedFsmAp::sortByStateId() +{ + /* Make the array. */ + int pos = 0; + RedStateAp **ptrList = new RedStateAp*[stateList.length()]; + for ( RedStateList::Iter st = stateList; st.lte(); st++, pos++ ) + ptrList[pos] = st; + + MergeSort<RedStateAp*, CmpStateById> mergeSort; + mergeSort.sort( ptrList, stateList.length() ); + + stateList.abandon(); + for ( int st = 0; st < pos; st++ ) + stateList.append( ptrList[st] ); + + delete[] ptrList; +} + +/* Find the final state with the lowest id. */ +void RedFsmAp::findFirstFinState() +{ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + if ( st->isFinal && (firstFinState == 0 || st->id < firstFinState->id) ) + firstFinState = st; + } +} + +void RedFsmAp::assignActionLocs() +{ + int nextLocation = 0; + for ( GenActionTableMap::Iter act = actionMap; act.lte(); act++ ) { + /* Store the loc, skip over the array and a null terminator. */ + act->location = nextLocation; + nextLocation += act->key.length() + 1; + } +} + +/* Check if we can extend the current range by displacing any ranges + * ahead to the singles. */ +bool RedFsmAp::canExtend( const RedTransList &list, int pos ) +{ + /* Get the transition that we want to extend. */ + RedTransAp *extendTrans = list[pos].value; + + /* Look ahead in the transition list. */ + for ( int next = pos + 1; next < list.length(); pos++, next++ ) { + /* If they are not continuous then cannot extend. */ + Key nextKey = list[next].lowKey; + nextKey.decrement(); + if ( list[pos].highKey != nextKey ) + break; + + /* Check for the extenstion property. */ + if ( extendTrans == list[next].value ) + return true; + + /* If the span of the next element is more than one, then don't keep + * checking, it won't be moved to single. */ + unsigned long long nextSpan = keyOps->span( list[next].lowKey, list[next].highKey ); + if ( nextSpan > 1 ) + break; + } + return false; +} + +/* Move ranges to the singles list. */ +void RedFsmAp::moveTransToSingle( RedStateAp *state ) +{ + RedTransList &range = state->outRange; + RedTransList &single = state->outSingle; + for ( int rpos = 0; rpos < range.length(); ) { + /* Check if this is a range we can extend. */ + if ( canExtend( range, rpos ) ) { + /* Transfer singles over. */ + while ( range[rpos].value != range[rpos+1].value ) { + /* Transfer the range to single. */ + single.append( range[rpos+1] ); + range.remove( rpos+1 ); + } + + /* Extend. */ + range[rpos].highKey = range[rpos+1].highKey; + range.remove( rpos+1 ); + } + /* Maybe move it to the singles. */ + else if ( keyOps->span( range[rpos].lowKey, range[rpos].highKey ) == 1 ) { + single.append( range[rpos] ); + range.remove( rpos ); + } + else { + /* Keeping it in the ranges. */ + rpos += 1; + } + } +} + +/* Look through ranges and choose suitable single character transitions. */ +void RedFsmAp::chooseSingle() +{ + /* Loop the states. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + /* Rewrite the transition list taking out the suitable single + * transtions. */ + moveTransToSingle( st ); + } +} + +void RedFsmAp::makeFlat() +{ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + if ( st->stateCondList.length() == 0 ) { + st->condLowKey = 0; + st->condHighKey = 0; + } + else { + st->condLowKey = st->stateCondList.head->lowKey; + st->condHighKey = st->stateCondList.tail->highKey; + + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + st->condList = new GenCondSpace*[ span ]; + memset( st->condList, 0, sizeof(GenCondSpace*)*span ); + + for ( GenStateCondList::Iter sci = st->stateCondList; sci.lte(); sci++ ) { + unsigned long long base, trSpan; + base = keyOps->span( st->condLowKey, sci->lowKey )-1; + trSpan = keyOps->span( sci->lowKey, sci->highKey ); + for ( unsigned long long pos = 0; pos < trSpan; pos++ ) + st->condList[base+pos] = sci->condSpace; + } + } + + if ( st->outRange.length() == 0 ) { + st->lowKey = st->highKey = 0; + st->transList = 0; + } + else { + st->lowKey = st->outRange[0].lowKey; + st->highKey = st->outRange[st->outRange.length()-1].highKey; + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + st->transList = new RedTransAp*[ span ]; + memset( st->transList, 0, sizeof(RedTransAp*)*span ); + + for ( RedTransList::Iter trans = st->outRange; trans.lte(); trans++ ) { + unsigned long long base, trSpan; + base = keyOps->span( st->lowKey, trans->lowKey )-1; + trSpan = keyOps->span( trans->lowKey, trans->highKey ); + for ( unsigned long long pos = 0; pos < trSpan; pos++ ) + st->transList[base+pos] = trans->value; + } + + /* Fill in the gaps with the default transition. */ + for ( unsigned long long pos = 0; pos < span; pos++ ) { + if ( st->transList[pos] == 0 ) + st->transList[pos] = st->defTrans; + } + } + } +} + + +/* A default transition has been picked, move it from the outRange to the + * default pointer. */ +void RedFsmAp::moveToDefault( RedTransAp *defTrans, RedStateAp *state ) +{ + /* Rewrite the outRange, omitting any ranges that use + * the picked default. */ + RedTransList outRange; + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) { + /* If it does not take the default, copy it over. */ + if ( rtel->value != defTrans ) + outRange.append( *rtel ); + } + + /* Save off the range we just created into the state's range. */ + state->outRange.transfer( outRange ); + + /* Store the default. */ + state->defTrans = defTrans; +} + +bool RedFsmAp::alphabetCovered( RedTransList &outRange ) +{ + /* Cannot cover without any out ranges. */ + if ( outRange.length() == 0 ) + return false; + + /* If the first range doesn't start at the the lower bound then the + * alphabet is not covered. */ + RedTransList::Iter rtel = outRange; + if ( keyOps->minKey < rtel->lowKey ) + return false; + + /* Check that every range is next to the previous one. */ + rtel.increment(); + for ( ; rtel.lte(); rtel++ ) { + Key highKey = rtel[-1].highKey; + highKey.increment(); + if ( highKey != rtel->lowKey ) + return false; + } + + /* The last must extend to the upper bound. */ + RedTransEl *last = &outRange[outRange.length()-1]; + if ( last->highKey < keyOps->maxKey ) + return false; + + return true; +} + +RedTransAp *RedFsmAp::chooseDefaultSpan( RedStateAp *state ) +{ + /* Make a set of transitions from the outRange. */ + RedTransSet stateTransSet; + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) + stateTransSet.insert( rtel->value ); + + /* For each transition in the find how many alphabet characters the + * transition spans. */ + unsigned long long *span = new unsigned long long[stateTransSet.length()]; + memset( span, 0, sizeof(unsigned long long) * stateTransSet.length() ); + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) { + /* Lookup the transition in the set. */ + RedTransAp **inSet = stateTransSet.find( rtel->value ); + int pos = inSet - stateTransSet.data; + span[pos] += keyOps->span( rtel->lowKey, rtel->highKey ); + } + + /* Find the max span, choose it for making the default. */ + RedTransAp *maxTrans = 0; + unsigned long long maxSpan = 0; + for ( RedTransSet::Iter rtel = stateTransSet; rtel.lte(); rtel++ ) { + if ( span[rtel.pos()] > maxSpan ) { + maxSpan = span[rtel.pos()]; + maxTrans = *rtel; + } + } + + delete[] span; + return maxTrans; +} + +/* Pick default transitions from ranges for the states. */ +void RedFsmAp::chooseDefaultSpan() +{ + /* Loop the states. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + /* Only pick a default transition if the alphabet is covered. This + * avoids any transitions in the out range that go to error and avoids + * the need for an ERR state. */ + if ( alphabetCovered( st->outRange ) ) { + /* Pick a default transition by largest span. */ + RedTransAp *defTrans = chooseDefaultSpan( st ); + + /* Rewrite the transition list taking out the transition we picked + * as the default and store the default. */ + moveToDefault( defTrans, st ); + } + } +} + +RedTransAp *RedFsmAp::chooseDefaultGoto( RedStateAp *state ) +{ + /* Make a set of transitions from the outRange. */ + RedTransSet stateTransSet; + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) { + if ( rtel->value->targ == state->next ) + return rtel->value; + } + return 0; +} + +void RedFsmAp::chooseDefaultGoto() +{ + /* Loop the states. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + /* Pick a default transition. */ + RedTransAp *defTrans = chooseDefaultGoto( st ); + if ( defTrans == 0 ) + defTrans = chooseDefaultSpan( st ); + + /* Rewrite the transition list taking out the transition we picked + * as the default and store the default. */ + moveToDefault( defTrans, st ); + } +} + +RedTransAp *RedFsmAp::chooseDefaultNumRanges( RedStateAp *state ) +{ + /* Make a set of transitions from the outRange. */ + RedTransSet stateTransSet; + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) + stateTransSet.insert( rtel->value ); + + /* For each transition in the find how many ranges use the transition. */ + int *numRanges = new int[stateTransSet.length()]; + memset( numRanges, 0, sizeof(int) * stateTransSet.length() ); + for ( RedTransList::Iter rtel = state->outRange; rtel.lte(); rtel++ ) { + /* Lookup the transition in the set. */ + RedTransAp **inSet = stateTransSet.find( rtel->value ); + numRanges[inSet - stateTransSet.data] += 1; + } + + /* Find the max number of ranges. */ + RedTransAp *maxTrans = 0; + int maxNumRanges = 0; + for ( RedTransSet::Iter rtel = stateTransSet; rtel.lte(); rtel++ ) { + if ( numRanges[rtel.pos()] > maxNumRanges ) { + maxNumRanges = numRanges[rtel.pos()]; + maxTrans = *rtel; + } + } + + delete[] numRanges; + return maxTrans; +} + +void RedFsmAp::chooseDefaultNumRanges() +{ + /* Loop the states. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + /* Pick a default transition. */ + RedTransAp *defTrans = chooseDefaultNumRanges( st ); + + /* Rewrite the transition list taking out the transition we picked + * as the default and store the default. */ + moveToDefault( defTrans, st ); + } +} + +RedTransAp *RedFsmAp::getErrorTrans( ) +{ + /* If the error trans has not been made aready, make it. */ + if ( errTrans == 0 ) { + /* This insert should always succeed since no transition created by + * the user can point to the error state. */ + errTrans = new RedTransAp( getErrorState(), 0, nextTransId++ ); + RedTransAp *inRes = transSet.insert( errTrans ); + assert( inRes != 0 ); + } + return errTrans; +} + +RedStateAp *RedFsmAp::getErrorState() +{ + /* Something went wrong. An error state is needed but one was not supplied + * by the frontend. */ + assert( errState != 0 ); + return errState; +} + + +RedTransAp *RedFsmAp::allocateTrans( RedStateAp *targ, RedAction *action ) +{ + /* Create a reduced trans and look for it in the transiton set. */ + RedTransAp redTrans( targ, action, 0 ); + RedTransAp *inDict = transSet.find( &redTrans ); + if ( inDict == 0 ) { + inDict = new RedTransAp( targ, action, nextTransId++ ); + transSet.insert( inDict ); + } + return inDict; +} + +void RedFsmAp::partitionFsm( int nparts ) +{ + /* At this point the states are ordered by a depth-first traversal. We + * will allocate to partitions based on this ordering. */ + this->nParts = nparts; + int partSize = stateList.length() / nparts; + int remainder = stateList.length() % nparts; + int numInPart = partSize; + int partition = 0; + if ( remainder-- > 0 ) + numInPart += 1; + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + st->partition = partition; + + numInPart -= 1; + if ( numInPart == 0 ) { + partition += 1; + numInPart = partSize; + if ( remainder-- > 0 ) + numInPart += 1; + } + } +} + +void RedFsmAp::setInTrans() +{ + /* First pass counts the number of transitions. */ + for ( TransApSet::Iter trans = transSet; trans.lte(); trans++ ) + trans->targ->numInTrans += 1; + + /* Pass over states to allocate the needed memory. Reset the counts so we + * can use them as the current size. */ + for ( RedStateList::Iter st = stateList; st.lte(); st++ ) { + st->inTrans = new RedTransAp*[st->numInTrans]; + st->numInTrans = 0; + } + + /* Second pass over transitions copies pointers into the in trans list. */ + for ( TransApSet::Iter trans = transSet; trans.lte(); trans++ ) + trans->targ->inTrans[trans->targ->numInTrans++] = trans; +} diff --git a/contrib/tools/ragel6/redfsm.h b/contrib/tools/ragel6/redfsm.h new file mode 100644 index 0000000000..badca23eb9 --- /dev/null +++ b/contrib/tools/ragel6/redfsm.h @@ -0,0 +1,532 @@ +/* + * Copyright 2001-2006 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _REDFSM_H +#define _REDFSM_H + +#include <assert.h> +#include <string.h> +#include <string> +#include "config.h" +#include "common.h" +#include "vector.h" +#include "dlist.h" +#include "compare.h" +#include "bstmap.h" +#include "bstset.h" +#include "avlmap.h" +#include "avltree.h" +#include "avlbasic.h" +#include "mergesort.h" +#include "sbstmap.h" +#include "sbstset.h" +#include "sbsttable.h" + + +#define TRANS_ERR_TRANS 0 +#define STATE_ERR_STATE 0 +#define FUNC_NO_FUNC 0 + +using std::string; + +struct RedStateAp; +struct GenInlineList; +struct GenAction; + +/* + * Inline code tree + */ +struct GenInlineItem +{ + enum Type + { + Text, Goto, Call, Next, GotoExpr, CallExpr, NextExpr, Ret, + PChar, Char, Hold, Exec, Curs, Targs, Entry, + LmSwitch, LmSetActId, LmSetTokEnd, LmGetTokEnd, LmInitTokStart, + LmInitAct, LmSetTokStart, SubAction, Break + }; + + GenInlineItem( const InputLoc &loc, Type type ) : + loc(loc), data(0), targId(0), targState(0), + lmId(0), children(0), offset(0), + type(type) { } + + InputLoc loc; + char *data; + int targId; + RedStateAp *targState; + int lmId; + GenInlineList *children; + int offset; + Type type; + + GenInlineItem *prev, *next; +}; + +/* Normally this would be atypedef, but that would entail including DList from + * ptreetypes, which should be just typedef forwards. */ +struct GenInlineList : public DList<GenInlineItem> { }; + +/* Element in list of actions. Contains the string for the code to exectute. */ +struct GenAction +: + public DListEl<GenAction> +{ + GenAction( ) + : + name(0), + inlineList(0), + actionId(0), + numTransRefs(0), + numToStateRefs(0), + numFromStateRefs(0), + numEofRefs(0) + { + } + + /* Data collected during parse. */ + InputLoc loc; + const char *name; + GenInlineList *inlineList; + int actionId; + + string nameOrLoc(); + + /* Number of references in the final machine. */ + int numRefs() + { return numTransRefs + numToStateRefs + numFromStateRefs + numEofRefs; } + int numTransRefs; + int numToStateRefs; + int numFromStateRefs; + int numEofRefs; +}; + + +/* Forwards. */ +struct RedStateAp; +struct StateAp; + +/* Transistion GenAction Element. */ +typedef SBstMapEl< int, GenAction* > GenActionTableEl; + +/* Transition GenAction Table. */ +struct GenActionTable + : public SBstMap< int, GenAction*, CmpOrd<int> > +{ + void setAction( int ordering, GenAction *action ); + void setActions( int *orderings, GenAction **actions, int nActs ); + void setActions( const GenActionTable &other ); +}; + +/* Compare of a whole action table element (key & value). */ +struct CmpGenActionTableEl +{ + static int compare( const GenActionTableEl &action1, + const GenActionTableEl &action2 ) + { + if ( action1.key < action2.key ) + return -1; + else if ( action1.key > action2.key ) + return 1; + else if ( action1.value < action2.value ) + return -1; + else if ( action1.value > action2.value ) + return 1; + return 0; + } +}; + +/* Compare for GenActionTable. */ +typedef CmpSTable< GenActionTableEl, CmpGenActionTableEl > CmpGenActionTable; + +/* Set of states. */ +typedef BstSet<RedStateAp*> RedStateSet; +typedef BstSet<int> IntSet; + +/* Reduced action. */ +struct RedAction +: + public AvlTreeEl<RedAction> +{ + RedAction( ) + : + key(), + eofRefs(0), + numTransRefs(0), + numToStateRefs(0), + numFromStateRefs(0), + numEofRefs(0), + bAnyNextStmt(false), + bAnyCurStateRef(false), + bAnyBreakStmt(false) + { } + + const GenActionTable &getKey() + { return key; } + + GenActionTable key; + int actListId; + int location; + IntSet *eofRefs; + + /* Number of references in the final machine. */ + int numRefs() + { return numTransRefs + numToStateRefs + numFromStateRefs + numEofRefs; } + int numTransRefs; + int numToStateRefs; + int numFromStateRefs; + int numEofRefs; + + bool anyNextStmt() { return bAnyNextStmt; } + bool anyCurStateRef() { return bAnyCurStateRef; } + bool anyBreakStmt() { return bAnyBreakStmt; } + + bool bAnyNextStmt; + bool bAnyCurStateRef; + bool bAnyBreakStmt; +}; +typedef AvlTree<RedAction, GenActionTable, CmpGenActionTable> GenActionTableMap; + +/* Reduced transition. */ +struct RedTransAp +: + public AvlTreeEl<RedTransAp> +{ + RedTransAp( RedStateAp *targ, RedAction *action, int id ) + : targ(targ), action(action), id(id), pos(-1), labelNeeded(true) { } + + RedStateAp *targ; + RedAction *action; + int id; + int pos; + bool partitionBoundary; + bool labelNeeded; +}; + +/* Compare of transitions for the final reduction of transitions. Comparison + * is on target and the pointer to the shared action table. It is assumed that + * when this is used the action tables have been reduced. */ +struct CmpRedTransAp +{ + static int compare( const RedTransAp &t1, const RedTransAp &t2 ) + { + if ( t1.targ < t2.targ ) + return -1; + else if ( t1.targ > t2.targ ) + return 1; + else if ( t1.action < t2.action ) + return -1; + else if ( t1.action > t2.action ) + return 1; + else + return 0; + } +}; + +typedef AvlBasic<RedTransAp, CmpRedTransAp> TransApSet; + +/* Element in out range. */ +struct RedTransEl +{ + /* Constructors. */ + RedTransEl( Key lowKey, Key highKey, RedTransAp *value ) + : lowKey(lowKey), highKey(highKey), value(value) { } + + Key lowKey, highKey; + RedTransAp *value; +}; + +typedef Vector<RedTransEl> RedTransList; +typedef Vector<RedStateAp*> RedStateVect; + +typedef BstMapEl<RedStateAp*, unsigned long long> RedSpanMapEl; +typedef BstMap<RedStateAp*, unsigned long long> RedSpanMap; + +/* Compare used by span map sort. Reverse sorts by the span. */ +struct CmpRedSpanMapEl +{ + static int compare( const RedSpanMapEl &smel1, const RedSpanMapEl &smel2 ) + { + if ( smel1.value > smel2.value ) + return -1; + else if ( smel1.value < smel2.value ) + return 1; + else + return 0; + } +}; + +/* Sorting state-span map entries by span. */ +typedef MergeSort<RedSpanMapEl, CmpRedSpanMapEl> RedSpanMapSort; + +/* Set of entry ids that go into this state. */ +typedef Vector<int> EntryIdVect; +typedef Vector<char*> EntryNameVect; + +typedef Vector< GenAction* > GenCondSet; + +struct Condition +{ + Condition( ) + : key(0), baseKey(0) {} + + Key key; + Key baseKey; + GenCondSet condSet; + + Condition *next, *prev; +}; +typedef DList<Condition> ConditionList; + +struct GenCondSpace +{ + Key baseKey; + GenCondSet condSet; + int condSpaceId; + + GenCondSpace *next, *prev; +}; +typedef DList<GenCondSpace> CondSpaceList; + +struct GenStateCond +{ + Key lowKey; + Key highKey; + + GenCondSpace *condSpace; + + GenStateCond *prev, *next; +}; +typedef DList<GenStateCond> GenStateCondList; +typedef Vector<GenStateCond*> StateCondVect; + +/* Reduced state. */ +struct RedStateAp +{ + RedStateAp() + : + defTrans(0), + condList(0), + transList(0), + isFinal(false), + labelNeeded(false), + outNeeded(false), + onStateList(false), + toStateAction(0), + fromStateAction(0), + eofAction(0), + eofTrans(0), + id(0), + bAnyRegCurStateRef(false), + partitionBoundary(false), + inTrans(0), + numInTrans(0) + { } + + /* Transitions out. */ + RedTransList outSingle; + RedTransList outRange; + RedTransAp *defTrans; + + /* For flat conditions. */ + Key condLowKey, condHighKey; + GenCondSpace **condList; + + /* For flat keys. */ + Key lowKey, highKey; + RedTransAp **transList; + + /* The list of states that transitions from this state go to. */ + RedStateVect targStates; + + bool isFinal; + bool labelNeeded; + bool outNeeded; + bool onStateList; + RedAction *toStateAction; + RedAction *fromStateAction; + RedAction *eofAction; + RedTransAp *eofTrans; + int id; + GenStateCondList stateCondList; + StateCondVect stateCondVect; + + /* Pointers for the list of states. */ + RedStateAp *prev, *next; + + bool anyRegCurStateRef() { return bAnyRegCurStateRef; } + bool bAnyRegCurStateRef; + + int partition; + bool partitionBoundary; + + RedTransAp **inTrans; + int numInTrans; +}; + +/* List of states. */ +typedef DList<RedStateAp> RedStateList; + +/* Set of reduced transitons. Comparison is by pointer. */ +typedef BstSet< RedTransAp*, CmpOrd<RedTransAp*> > RedTransSet; + +/* Next version of the fsm machine. */ +struct RedFsmAp +{ + RedFsmAp(); + + bool forcedErrorState; + + int nextActionId; + int nextTransId; + + /* Next State Id doubles as the total number of state ids. */ + int nextStateId; + + TransApSet transSet; + GenActionTableMap actionMap; + RedStateList stateList; + RedStateSet entryPoints; + RedStateAp *startState; + RedStateAp *errState; + RedTransAp *errTrans; + RedTransAp *errActionTrans; + RedStateAp *firstFinState; + int numFinStates; + int nParts; + + bool bAnyToStateActions; + bool bAnyFromStateActions; + bool bAnyRegActions; + bool bAnyEofActions; + bool bAnyEofTrans; + bool bAnyActionGotos; + bool bAnyActionCalls; + bool bAnyActionRets; + bool bAnyActionByValControl; + bool bAnyRegActionRets; + bool bAnyRegActionByValControl; + bool bAnyRegNextStmt; + bool bAnyRegCurStateRef; + bool bAnyRegBreak; + bool bAnyConditions; + + int maxState; + int maxSingleLen; + int maxRangeLen; + int maxKeyOffset; + int maxIndexOffset; + int maxIndex; + int maxActListId; + int maxActionLoc; + int maxActArrItem; + unsigned long long maxSpan; + unsigned long long maxCondSpan; + int maxFlatIndexOffset; + Key maxKey; + int maxCondOffset; + int maxCondLen; + int maxCondSpaceId; + int maxCondIndexOffset; + int maxCond; + + bool anyActions(); + bool anyToStateActions() { return bAnyToStateActions; } + bool anyFromStateActions() { return bAnyFromStateActions; } + bool anyRegActions() { return bAnyRegActions; } + bool anyEofActions() { return bAnyEofActions; } + bool anyEofTrans() { return bAnyEofTrans; } + bool anyActionGotos() { return bAnyActionGotos; } + bool anyActionCalls() { return bAnyActionCalls; } + bool anyActionRets() { return bAnyActionRets; } + bool anyActionByValControl() { return bAnyActionByValControl; } + bool anyRegActionRets() { return bAnyRegActionRets; } + bool anyRegActionByValControl() { return bAnyRegActionByValControl; } + bool anyRegNextStmt() { return bAnyRegNextStmt; } + bool anyRegCurStateRef() { return bAnyRegCurStateRef; } + bool anyRegBreak() { return bAnyRegBreak; } + bool anyConditions() { return bAnyConditions; } + + + /* Is is it possible to extend a range by bumping ranges that span only + * one character to the singles array. */ + bool canExtend( const RedTransList &list, int pos ); + + /* Pick single transitions from the ranges. */ + void moveTransToSingle( RedStateAp *state ); + void chooseSingle(); + + void makeFlat(); + + /* Move a selected transition from ranges to default. */ + void moveToDefault( RedTransAp *defTrans, RedStateAp *state ); + + /* Pick a default transition by largest span. */ + RedTransAp *chooseDefaultSpan( RedStateAp *state ); + void chooseDefaultSpan(); + + /* Pick a default transition by most number of ranges. */ + RedTransAp *chooseDefaultNumRanges( RedStateAp *state ); + void chooseDefaultNumRanges(); + + /* Pick a default transition tailored towards goto driven machine. */ + RedTransAp *chooseDefaultGoto( RedStateAp *state ); + void chooseDefaultGoto(); + + /* Ordering states by transition connections. */ + void optimizeStateOrdering( RedStateAp *state ); + void optimizeStateOrdering(); + + /* Ordering states by transition connections. */ + void depthFirstOrdering( RedStateAp *state ); + void depthFirstOrdering(); + + /* Set state ids. */ + void sequentialStateIds(); + void sortStateIdsByFinal(); + + /* Arrange states in by final id. This is a stable sort. */ + void sortStatesByFinal(); + + /* Sorting states by id. */ + void sortByStateId(); + + /* Locating the first final state. This is the final state with the lowest + * id. */ + void findFirstFinState(); + + void assignActionLocs(); + + RedTransAp *getErrorTrans(); + RedStateAp *getErrorState(); + + /* Is every char in the alphabet covered? */ + bool alphabetCovered( RedTransList &outRange ); + + RedTransAp *allocateTrans( RedStateAp *targState, RedAction *actionTable ); + + void partitionFsm( int nParts ); + + void setInTrans(); +}; + + +#endif diff --git a/contrib/tools/ragel6/rlparse.cpp b/contrib/tools/ragel6/rlparse.cpp new file mode 100644 index 0000000000..35b3fff054 --- /dev/null +++ b/contrib/tools/ragel6/rlparse.cpp @@ -0,0 +1,6934 @@ +/* Automatically generated by Kelbt from "rlparse.kl". + * + * Parts of this file are copied from Kelbt source covered by the GNU + * GPL. As a special exception, you may use the parts of this file copied + * from Kelbt source without restriction. The remainder is derived from + * "rlparse.kl" and inherits the copyright status of that file. + */ + +#line 1 "rlparse.kl" +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "rlparse.h" +#include "ragel.h" +#include <iostream> +#include <errno.h> +#include <stdlib.h> + +using std::cout; +using std::cerr; +using std::endl; + +#line 102 "rlparse.kh" +#line 105 "rlparse.kh" +#line 140 "rlparse.kh" +#line 1444 "rlparse.kl" + + +#line 48 "rlparse.cpp" +struct Parser_Lel_action_ref +{ +#line 755 "rlparse.kl" + + Action *action; + + +#line 54 "rlparse.cpp" +}; + +struct Parser_Lel_aug_type +{ +#line 546 "rlparse.kl" + + InputLoc loc; + AugType augType; + + +#line 65 "rlparse.cpp" +}; + +struct Parser_Lel_expression +{ +#line 338 "rlparse.kl" + + Expression *expression; + + +#line 75 "rlparse.cpp" +}; + +struct Parser_Lel_factor +{ +#line 975 "rlparse.kl" + + Factor *factor; + + +#line 85 "rlparse.cpp" +}; + +struct Parser_Lel_factor_rep_num +{ +#line 929 "rlparse.kl" + + int rep; + + +#line 95 "rlparse.cpp" +}; + +struct Parser_Lel_factor_with_aug +{ +#line 452 "rlparse.kl" + + FactorWithAug *factorWithAug; + + +#line 105 "rlparse.cpp" +}; + +struct Parser_Lel_factor_with_ep +{ +#line 436 "rlparse.kl" + + FactorWithAug *factorWithAug; + + +#line 115 "rlparse.cpp" +}; + +struct Parser_Lel_factor_with_label +{ +#line 420 "rlparse.kl" + + FactorWithAug *factorWithAug; + + +#line 125 "rlparse.cpp" +}; + +struct Parser_Lel_factor_with_neg +{ +#line 939 "rlparse.kl" + + FactorWithNeg *factorWithNeg; + + +#line 135 "rlparse.cpp" +}; + +struct Parser_Lel_factor_with_rep +{ +#line 868 "rlparse.kl" + + FactorWithRep *factorWithRep; + + +#line 145 "rlparse.cpp" +}; + +struct Parser_Lel_inline_item +{ +#line 1234 "rlparse.kl" + + InlineItem *inlineItem; + + +#line 155 "rlparse.cpp" +}; + +struct Parser_Lel_inline_list +{ +#line 1213 "rlparse.kl" + + InlineList *inlineList; + + +#line 165 "rlparse.cpp" +}; + +struct Parser_Lel_join +{ +#line 321 "rlparse.kl" + + Join *join; + + +#line 175 "rlparse.cpp" +}; + +struct Parser_Lel_join_or_lm +{ +#line 229 "rlparse.kl" + + MachineDef *machineDef; + + +#line 185 "rlparse.cpp" +}; + +struct Parser_Lel_lm_part_list +{ +#line 253 "rlparse.kl" + + LmPartList *lmPartList; + + +#line 195 "rlparse.cpp" +}; + +struct Parser_Lel_local_err_name +{ +#line 856 "rlparse.kl" + + int error_name; + + +#line 205 "rlparse.cpp" +}; + +struct Parser_Lel_longest_match_part +{ +#line 277 "rlparse.kl" + + LongestMatchPart *lmPart; + + +#line 215 "rlparse.cpp" +}; + +struct Parser_Lel_opt_export +{ +#line 95 "rlparse.kl" + + bool isSet; + + +#line 225 "rlparse.cpp" +}; + +struct Parser_Lel_opt_lm_part_action +{ +#line 294 "rlparse.kl" + + Action *action; + + +#line 235 "rlparse.cpp" +}; + +struct Parser_Lel_priority_aug +{ +#line 803 "rlparse.kl" + + int priorityNum; + + +#line 245 "rlparse.cpp" +}; + +struct Parser_Lel_priority_name +{ +#line 788 "rlparse.kl" + + int priorityName; + + +#line 255 "rlparse.cpp" +}; + +struct Parser_Lel_range_lit +{ +#line 1042 "rlparse.kl" + + Literal *literal; + + +#line 265 "rlparse.cpp" +}; + +struct Parser_Lel_regular_expr +{ +#line 1079 "rlparse.kl" + + RegExpr *regExpr; + + +#line 275 "rlparse.cpp" +}; + +struct Parser_Lel_regular_expr_char +{ +#line 1131 "rlparse.kl" + + ReItem *reItem; + + +#line 285 "rlparse.cpp" +}; + +struct Parser_Lel_regular_expr_item +{ +#line 1114 "rlparse.kl" + + ReItem *reItem; + + +#line 295 "rlparse.cpp" +}; + +struct Parser_Lel_regular_expr_or_char +{ +#line 1188 "rlparse.kl" + + ReOrItem *reOrItem; + + +#line 305 "rlparse.cpp" +}; + +struct Parser_Lel_regular_expr_or_data +{ +#line 1155 "rlparse.kl" + + ReOrBlock *reOrBlock; + + +#line 315 "rlparse.cpp" +}; + +struct Parser_Lel_term +{ +#line 389 "rlparse.kl" + + Term *term; + + +#line 325 "rlparse.cpp" +}; + +struct Parser_Lel_term_short +{ +#line 368 "rlparse.kl" + + Term *term; + + +#line 335 "rlparse.cpp" +}; + +struct Parser_Lel_token_type +{ +#line 146 "rlparse.kl" + + Token token; + + +#line 345 "rlparse.cpp" +}; + +union Parser_UserData +{ + struct Parser_Lel_action_ref action_ref; + struct Parser_Lel_aug_type aug_type; + struct Parser_Lel_expression expression; + struct Parser_Lel_factor factor; + struct Parser_Lel_factor_rep_num factor_rep_num; + struct Parser_Lel_factor_with_aug factor_with_aug; + struct Parser_Lel_factor_with_ep factor_with_ep; + struct Parser_Lel_factor_with_label factor_with_label; + struct Parser_Lel_factor_with_neg factor_with_neg; + struct Parser_Lel_factor_with_rep factor_with_rep; + struct Parser_Lel_inline_item inline_item; + struct Parser_Lel_inline_list inline_list; + struct Parser_Lel_join join; + struct Parser_Lel_join_or_lm join_or_lm; + struct Parser_Lel_lm_part_list lm_part_list; + struct Parser_Lel_local_err_name local_err_name; + struct Parser_Lel_longest_match_part longest_match_part; + struct Parser_Lel_opt_export opt_export; + struct Parser_Lel_opt_lm_part_action opt_lm_part_action; + struct Parser_Lel_priority_aug priority_aug; + struct Parser_Lel_priority_name priority_name; + struct Parser_Lel_range_lit range_lit; + struct Parser_Lel_regular_expr regular_expr; + struct Parser_Lel_regular_expr_char regular_expr_char; + struct Parser_Lel_regular_expr_item regular_expr_item; + struct Parser_Lel_regular_expr_or_char regular_expr_or_char; + struct Parser_Lel_regular_expr_or_data regular_expr_or_data; + struct Parser_Lel_term term; + struct Parser_Lel_term_short term_short; + struct Parser_Lel_token_type token_type; + struct Token token; +}; + +struct Parser_LangEl +{ + char *file; + int line; + int type; + int reduction; + int state; + int causeReduce; + union Parser_UserData user; + unsigned int retry; + struct Parser_LangEl *next, *child, *prev; +}; + +struct Parser_Block +{ + struct Parser_LangEl data[8128]; + struct Parser_Block *next; +}; + +#line 404 "rlparse.cpp" +unsigned int Parser_startState = 0; + +short Parser_indicies[] = { + 152, -1, -1, 152, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 152, 152, 152, 152, -1, + -1, -1, -1, -1, -1, -1, -1, 152, + 152, 152, 152, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 152, + 152, 152, 1, 0, 404, 154, -1, -1, + 154, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 154, 154, 154, 154, -1, -1, -1, -1, + -1, -1, -1, -1, 154, 154, 154, 154, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 154, 154, 150, -1, + -1, 2, 161, -1, -1, 151, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 4, 5, 6, + 7, -1, -1, -1, -1, -1, -1, -1, + -1, 158, 11, 12, 13, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 9, 8, -1, -1, -1, -1, 153, + 392, 393, 394, 395, 396, 397, 398, 399, + 400, 401, 402, 403, -1, 10, 3, 165, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 26, 14, 15, 17, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 326, 328, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 16, 364, 364, 364, -1, 364, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 364, -1, -1, -1, 364, + 364, -1, -1, -1, -1, -1, -1, -1, + -1, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, 364, + 364, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, 364, -1, -1, -1, 364, 364, + 364, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 19, 364, 364, 364, + -1, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 364, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 364, -1, -1, + -1, 364, 364, -1, -1, -1, -1, -1, + -1, -1, -1, 364, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 364, 364, 364, 364, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 364, 364, -1, -1, -1, + 364, 364, 364, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 24, 174, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 174, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 23, 25, -1, -1, -1, -1, 159, + 20, 21, 22, 27, 168, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 28, 17, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 326, 328, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 29, 327, 376, + 377, 378, -1, 375, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 170, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 374, + -1, -1, -1, 372, 373, -1, -1, -1, + -1, -1, -1, -1, -1, 379, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 368, 369, 370, 371, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 380, 381, -1, + -1, -1, 382, 383, 30, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 365, -1, + 367, -1, 363, 366, 342, 342, 342, -1, + 342, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 342, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 342, -1, -1, 342, -1, -1, -1, + 342, 342, -1, -1, -1, -1, -1, -1, + -1, -1, 342, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 342, + 342, 342, 342, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 342, + 342, 342, 342, 342, 342, 342, 342, 342, + 342, 342, 342, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 31, 342, + 342, 342, -1, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 342, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 342, -1, -1, 342, + -1, -1, -1, 342, 342, -1, -1, -1, + -1, -1, -1, -1, -1, 342, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 342, 342, 342, 342, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 342, 342, 342, 342, 342, 342, + 342, 342, 342, 342, 342, 342, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 32, 155, 33, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 173, 376, + 377, 378, -1, 375, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 171, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 374, + -1, -1, -1, 372, 373, -1, -1, -1, + -1, -1, -1, -1, -1, 379, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 368, 369, 370, 371, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 380, 381, -1, + -1, -1, 382, 383, 30, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 365, -1, + 367, -1, 363, 366, 154, -1, -1, 154, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 154, 154, 154, 154, -1, -1, + -1, -1, -1, -1, -1, -1, 154, 154, + 154, 154, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 154, 154, + -1, -1, -1, 34, 35, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 36, 342, 342, 342, + -1, 342, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 342, -1, -1, 342, -1, -1, + -1, 342, 342, -1, -1, -1, -1, -1, + -1, -1, -1, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, 342, 342, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, 342, 342, 342, 342, 342, 342, 342, + 342, 342, 342, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 37, + 167, 169, 38, 348, 349, 350, -1, 346, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 347, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 156, -1, -1, 374, -1, -1, -1, 372, + 373, -1, -1, -1, -1, -1, -1, -1, + -1, 351, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 368, 369, + 370, 371, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 45, 40, + 39, 380, 381, 41, 43, 44, 382, 383, + 30, 42, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 341, 345, 343, 344, 352, 348, 349, 350, + -1, 346, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 347, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 157, -1, -1, 374, -1, -1, + -1, 372, 373, -1, -1, -1, -1, -1, + -1, -1, -1, 351, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 368, 369, 370, 371, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 45, 40, 39, 380, 381, 41, 43, 44, + 382, 383, 30, 42, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 341, 345, 343, 344, 352, 364, + 364, 364, -1, 364, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 364, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 364, + -1, -1, -1, 364, 364, -1, -1, -1, + -1, -1, -1, -1, -1, 364, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 364, 364, 364, 364, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 364, 364, -1, + -1, -1, 364, 364, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 46, 162, -1, -1, 161, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 4, + 5, 6, 7, -1, -1, -1, -1, -1, + -1, -1, -1, 158, 11, 12, 13, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 9, 8, -1, -1, -1, + -1, 153, 392, 393, 394, 395, 396, 397, + 398, 399, 400, 401, 402, 403, -1, 10, + 3, 55, -1, -1, -1, -1, -1, -1, + 63, -1, -1, -1, -1, 17, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 56, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 50, 57, -1, -1, 326, 328, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 61, 59, 60, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 47, -1, + 58, -1, -1, -1, -1, -1, -1, -1, + 48, 191, 49, 198, 52, -1, 53, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 54, -1, -1, -1, 308, 312, -1, + -1, 62, 55, -1, -1, -1, -1, -1, + -1, 63, -1, -1, -1, -1, 17, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 56, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 50, 57, -1, -1, 326, 328, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 66, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 61, 59, 60, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 65, + 64, 58, -1, -1, -1, -1, -1, -1, + -1, 48, 191, 49, 198, 52, -1, 53, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 54, -1, -1, -1, 308, 312, + -1, -1, 62, 348, 349, 350, -1, 346, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 347, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 166, -1, -1, 374, -1, -1, -1, 372, + 373, -1, -1, -1, -1, -1, -1, -1, + -1, 351, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 368, 369, + 370, 371, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 45, 40, + 39, 380, 381, 41, 43, 44, 382, 383, + 30, 42, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 341, 345, 343, 344, 352, 389, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 388, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 67, + -1, -1, -1, -1, 68, 353, 364, 364, + 364, -1, 364, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, -1, + -1, -1, 364, 364, -1, -1, -1, -1, + -1, -1, -1, -1, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, 364, 364, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 364, 364, -1, -1, + -1, 364, 364, 364, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 69, + 71, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 389, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 388, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 70, -1, -1, -1, -1, 68, 75, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 389, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 388, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 74, -1, + -1, -1, -1, 68, 73, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 389, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 388, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 72, -1, -1, -1, + -1, 68, 361, 362, 376, 377, 378, -1, + 375, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 172, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 374, -1, -1, -1, + 372, 373, -1, -1, -1, -1, -1, -1, + -1, -1, 379, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 368, + 369, 370, 371, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 380, 381, -1, -1, -1, 382, + 383, 30, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 365, -1, 367, -1, 363, + 366, 81, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 163, 83, -1, -1, 186, -1, -1, 186, + 84, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 186, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 186, 82, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 186, -1, + -1, -1, -1, 85, 55, -1, -1, -1, + -1, 192, -1, 63, 192, -1, -1, 192, + 18, 86, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 192, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 56, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 192, 192, + -1, -1, -1, 50, 57, -1, -1, 326, + 328, -1, 87, 88, 89, -1, 192, -1, + -1, -1, -1, 192, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 61, 59, + 60, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 58, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 193, 52, + -1, 53, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 54, -1, -1, -1, + 308, 312, -1, -1, 62, 315, -1, -1, + 315, 315, 315, -1, 315, 315, 315, 315, + 315, 315, 315, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 77, 315, + 315, -1, 315, 315, 315, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 315, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 315, + 315, -1, -1, -1, 315, 315, -1, -1, + 315, 315, -1, 315, 315, 315, 315, 315, + 315, -1, -1, -1, 315, 315, 315, 315, + 315, 315, 315, 315, 315, 315, 315, 315, + 315, 315, 315, 315, 315, 315, 315, 315, + 315, 315, 315, 315, 315, 315, 315, 315, + 315, 315, 315, 315, 315, 315, 315, 315, + 315, 315, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 315, 315, 315, 200, -1, + -1, -1, -1, 200, -1, 200, 200, -1, + -1, 200, 200, 200, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 200, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 200, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 200, 200, -1, -1, -1, 200, 200, -1, + -1, 200, 200, -1, 200, 200, 200, 90, + 200, -1, -1, -1, -1, 200, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 200, 200, 200, 202, -1, -1, 100, 99, + 202, -1, 202, 202, -1, -1, 202, 202, + 202, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 202, 102, -1, + 101, -1, 98, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 202, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 202, 202, -1, + -1, -1, 202, 202, -1, -1, 202, 202, + -1, 202, 202, 202, 202, 202, -1, -1, + -1, -1, 202, 219, 221, 223, 103, 264, + 268, 270, 272, 266, 274, 276, 280, 282, + 284, 278, 286, 252, 256, 258, 260, 254, + 262, 228, 232, 234, 236, 230, 238, 240, + 244, 246, 248, 242, 250, 202, 202, 202, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 226, 225, 227, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 91, -1, -1, 92, 93, 94, 95, + 96, 97, 214, -1, -1, 214, 214, 214, + -1, 214, 214, 300, 303, 214, 214, 214, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 214, 214, -1, 214, + 302, 214, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 214, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 104, 214, -1, -1, + -1, 214, 214, -1, -1, 214, 214, -1, + 214, 214, 214, 214, 214, 301, -1, -1, + -1, 214, 214, 214, 214, 214, 214, 214, + 214, 214, 214, 214, 214, 214, 214, 214, + 214, 214, 214, 214, 214, 214, 214, 214, + 214, 214, 214, 214, 214, 214, 214, 214, + 214, 214, 214, 214, 214, 214, 214, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 214, 214, 214, 55, -1, -1, -1, -1, + -1, -1, 63, -1, -1, -1, -1, 17, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 56, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 316, 57, -1, -1, 326, 328, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 61, 59, 60, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 58, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 310, + 312, -1, -1, 62, 55, -1, -1, -1, + -1, -1, -1, 63, -1, -1, -1, -1, + 17, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 56, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 316, 57, -1, -1, 326, + 328, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 61, 59, + 60, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 58, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 311, 312, -1, -1, 62, 313, -1, -1, + 313, 313, 313, -1, 313, 313, 313, 313, + 313, 313, 313, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 313, + 313, -1, 313, 313, 313, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 313, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 313, + 313, -1, -1, -1, 313, 313, -1, -1, + 313, 313, 322, 313, 313, 313, 313, 313, + 313, -1, -1, -1, 313, 313, 313, 313, + 313, 313, 313, 313, 313, 313, 313, 313, + 313, 313, 313, 313, 313, 313, 313, 313, + 313, 313, 313, 313, 313, 313, 313, 313, + 313, 313, 313, 313, 313, 313, 313, 313, + 313, 313, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 313, 313, 313, 314, -1, + -1, 314, 314, 314, -1, 314, 314, 314, + 314, 314, 314, 314, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 314, 314, -1, 314, 314, 314, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 314, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 314, 314, -1, -1, -1, 314, 314, -1, + -1, 314, 314, 324, 314, 314, 314, 314, + 314, 314, -1, -1, -1, 314, 314, 314, + 314, 314, 314, 314, 314, 314, 314, 314, + 314, 314, 314, 314, 314, 314, 314, 314, + 314, 314, 314, 314, 314, 314, 314, 314, + 314, 314, 314, 314, 314, 314, 314, 314, + 314, 314, 314, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 314, 314, 314, 338, + -1, -1, -1, 338, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 79, 338, -1, -1, -1, 338, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 80, 330, 330, 330, -1, 330, + -1, -1, 330, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 78, 105, 55, -1, -1, -1, -1, -1, + -1, 63, -1, -1, -1, -1, 17, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 56, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 50, 57, -1, -1, 326, 328, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 61, 59, 60, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 76, + -1, 58, -1, -1, -1, -1, -1, -1, + -1, 48, 191, 49, 198, 52, -1, 53, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 54, -1, -1, -1, 308, 312, + -1, -1, 62, 164, 81, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 175, 55, -1, -1, -1, + -1, -1, -1, 63, -1, -1, -1, -1, + 17, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 56, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 51, 57, -1, -1, 326, + 328, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 61, 59, + 60, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 4, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 158, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 180, -1, 179, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 160, 108, + -1, 107, -1, 58, -1, -1, 106, 178, + -1, -1, -1, 48, 191, 49, 198, 52, + -1, 53, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 54, -1, -1, -1, + 308, 312, -1, -1, 62, 384, 391, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 109, 376, 377, 378, + -1, 375, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 354, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 374, -1, -1, + -1, 372, 373, -1, -1, -1, -1, -1, + -1, -1, -1, 379, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 368, 369, 370, 371, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 380, 381, -1, -1, -1, + 382, 383, 30, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 365, -1, 367, -1, + 363, 366, 355, 364, 364, 364, -1, 364, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 364, -1, -1, -1, 364, + 364, -1, -1, -1, -1, -1, -1, -1, + -1, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, 364, + 364, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, 364, -1, -1, -1, 364, 364, + 364, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 110, 359, 364, 364, + 364, -1, 364, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 364, -1, + -1, -1, 364, 364, -1, -1, -1, -1, + -1, -1, -1, -1, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, 364, 364, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 364, 364, -1, -1, + -1, 364, 364, 364, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 111, + 357, 364, 364, 364, -1, 364, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 364, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 364, -1, -1, -1, 364, 364, -1, + -1, -1, -1, -1, -1, -1, -1, 364, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 364, 364, 364, 364, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 364, + 364, -1, -1, -1, 364, 364, 364, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 112, 321, -1, -1, 81, 55, + -1, -1, -1, -1, -1, -1, 63, -1, + -1, -1, -1, 17, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 56, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 50, 57, + -1, -1, 326, 328, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 61, 59, 60, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 58, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 199, 52, -1, 53, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 54, + -1, -1, -1, 308, 312, -1, -1, 62, + 319, 114, 115, -1, 335, -1, -1, 336, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 329, + 113, 317, -1, -1, -1, 116, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 337, + 318, -1, -1, -1, 116, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 337, 55, + -1, -1, -1, -1, -1, -1, 63, -1, + -1, -1, -1, 17, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 56, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 50, 57, + -1, -1, 326, 328, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 61, 59, 60, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 58, -1, + -1, -1, -1, -1, -1, -1, 117, 191, + 49, 198, 52, -1, 53, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 54, + -1, -1, -1, 308, 312, -1, -1, 62, + 55, -1, -1, -1, -1, -1, -1, 63, + -1, -1, -1, -1, 17, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 56, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 50, + 57, -1, -1, 326, 328, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 61, 59, 60, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 58, + -1, -1, -1, -1, -1, -1, -1, -1, + 187, 49, 198, 52, -1, 53, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 54, -1, -1, -1, 308, 312, -1, -1, + 62, 55, -1, -1, -1, -1, -1, -1, + 63, -1, -1, -1, -1, 17, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 56, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 50, 57, -1, -1, 326, 328, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 61, 59, 60, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 58, -1, -1, -1, -1, -1, -1, -1, + -1, 188, 49, 198, 52, -1, 53, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 54, -1, -1, -1, 308, 312, -1, + -1, 62, 55, -1, -1, -1, -1, -1, + -1, 63, -1, -1, -1, -1, 17, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 56, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 50, 57, -1, -1, 326, 328, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 61, 59, 60, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 58, -1, -1, -1, -1, -1, -1, + -1, -1, 189, 49, 198, 52, -1, 53, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 54, -1, -1, -1, 308, 312, + -1, -1, 62, 55, -1, -1, -1, -1, + -1, -1, 63, -1, -1, -1, -1, 17, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 56, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 50, 57, -1, -1, 326, 328, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 61, 59, 60, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 58, -1, -1, -1, -1, -1, + -1, -1, -1, 190, 49, 198, 52, -1, + 53, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 54, -1, -1, -1, 308, + 312, -1, -1, 62, 55, -1, -1, -1, + -1, -1, -1, 63, -1, -1, -1, -1, + 17, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 56, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 50, 57, -1, -1, 326, + 328, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 61, 59, + 60, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 58, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 194, 52, + -1, 53, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 54, -1, -1, -1, + 308, 312, -1, -1, 62, 55, -1, -1, + -1, -1, -1, -1, 63, -1, -1, -1, + -1, 17, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 56, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 50, 57, -1, -1, + 326, 328, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 61, + 59, 60, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 58, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 195, + 52, -1, 53, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 54, -1, -1, + -1, 308, 312, -1, -1, 62, 55, -1, + -1, -1, -1, -1, -1, 63, -1, -1, + -1, -1, 17, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 56, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 50, 57, -1, + -1, 326, 328, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 61, 59, 60, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 58, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 196, 52, -1, 53, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 54, -1, + -1, -1, 308, 312, -1, -1, 62, 55, + -1, -1, -1, -1, -1, -1, 63, -1, + -1, -1, -1, 17, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 56, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 50, 57, + -1, -1, 326, 328, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 61, 59, 60, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 58, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 197, 52, -1, 53, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 54, + -1, -1, -1, 308, 312, -1, -1, 62, + 386, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 201, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 118, 119, -1, + -1, 121, -1, 122, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 120, -1, -1, -1, -1, 292, -1, + -1, -1, 296, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 203, 290, -1, -1, + -1, -1, -1, -1, -1, -1, 204, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 288, 295, 123, -1, -1, -1, -1, -1, + -1, 124, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 120, -1, -1, -1, + -1, 292, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 206, + 290, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 288, 124, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 120, + -1, -1, -1, -1, 292, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 208, 290, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 288, 124, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 120, -1, -1, -1, -1, 292, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 209, 290, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 288, 124, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 120, -1, -1, + -1, -1, 292, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 210, 290, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 288, 124, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 120, -1, -1, -1, -1, 292, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 211, 290, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 288, + 125, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 120, -1, -1, -1, -1, + 292, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 212, 290, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 288, 215, -1, -1, 215, -1, + 215, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 215, -1, + -1, -1, -1, 215, -1, -1, -1, 215, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 261, 273, 285, + 237, 249, 216, -1, -1, 216, -1, 216, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 216, -1, -1, + -1, -1, 216, -1, -1, -1, 216, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 224, -1, 259, 271, 283, 235, + 247, 217, -1, -1, 217, -1, 217, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 217, -1, -1, -1, + -1, 217, -1, -1, -1, 217, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 222, -1, 257, 269, 281, 233, 245, + 218, -1, -1, 218, -1, 218, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 218, -1, -1, -1, -1, + 218, -1, -1, -1, 218, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 220, -1, 253, 265, 277, 229, 241, 255, + 267, 279, 231, 243, 263, 275, 287, 239, + 251, 127, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 309, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 126, 17, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 323, -1, + -1, 326, 328, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 325, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 320, 55, + -1, -1, -1, -1, -1, -1, 63, -1, + 131, -1, -1, 17, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 56, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 51, 57, + -1, -1, 326, 328, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 61, 59, 60, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 4, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 158, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 180, -1, 179, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 160, 108, -1, 107, -1, 58, -1, + -1, -1, 177, -1, -1, -1, 48, 191, + 49, 198, 52, -1, 53, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 54, + -1, -1, -1, 308, 312, -1, -1, 62, + 81, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 184, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 120, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 130, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 129, -1, 183, 165, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 128, 387, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 387, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 132, + 376, 377, 378, -1, 375, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 356, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 374, -1, -1, -1, 372, 373, -1, -1, + -1, -1, -1, -1, -1, -1, 379, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 368, 369, 370, 371, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 380, 381, + -1, -1, -1, 382, 383, 30, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 365, + -1, 367, -1, 363, 366, 376, 377, 378, + -1, 375, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 360, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 374, -1, -1, + -1, 372, 373, -1, -1, -1, -1, -1, + -1, -1, -1, 379, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 368, 369, 370, 371, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 380, 381, -1, -1, -1, + 382, 383, 30, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 365, -1, 367, -1, + 363, 366, 376, 377, 378, -1, 375, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 358, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 374, -1, -1, -1, 372, 373, + -1, -1, -1, -1, -1, -1, -1, -1, + 379, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 368, 369, 370, + 371, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 380, 381, -1, -1, -1, 382, 383, 30, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 365, -1, 367, -1, 363, 366, 332, + 332, 332, -1, 332, 331, -1, 332, 338, + -1, -1, -1, 338, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 133, 338, -1, -1, -1, 338, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 134, 339, -1, -1, 135, 339, + 83, -1, -1, 185, -1, -1, 185, 84, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 185, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 185, 82, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 185, -1, -1, + -1, -1, 85, 391, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 136, 139, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 137, -1, -1, -1, -1, + -1, -1, -1, -1, 138, 342, 342, 342, + -1, 342, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 342, -1, -1, 342, -1, -1, + -1, 342, 342, -1, -1, -1, -1, -1, + -1, -1, -1, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, 342, 342, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 342, 342, 342, 342, 342, 342, 342, 342, + 342, 342, 342, 342, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 140, + 297, 298, 124, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 120, -1, -1, + -1, -1, 292, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 207, 290, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 288, 292, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 138, + 142, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 141, -1, 138, 144, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 304, 309, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, 143, 35, 181, + 124, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 120, -1, -1, -1, -1, + 292, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 182, 290, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 288, 176, 390, 333, -1, -1, + -1, 116, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 337, 334, -1, -1, -1, + 116, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 337, 340, 385, -1, -1, -1, + -1, 385, -1, 385, 385, -1, -1, 385, + 385, 385, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 385, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, 385, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 385, 385, + -1, -1, -1, 385, 385, -1, -1, 385, + 385, -1, 385, 385, 385, 385, 385, -1, + -1, 132, -1, 385, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 385, 385, + 385, 145, 289, 291, -1, -1, 294, 348, + 349, 350, -1, 346, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 347, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 293, -1, -1, 374, + -1, -1, -1, 372, 373, -1, -1, -1, + -1, -1, -1, -1, -1, 351, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 368, 369, 370, 371, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 45, 40, 39, 380, 381, 41, + 43, 44, 382, 383, 30, 42, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 341, 345, 343, 344, + 352, 146, 291, -1, -1, 299, 305, 306, + -1, -1, -1, -1, -1, -1, 309, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 147, 121, + -1, 122, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + 296, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, 148, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, 295, + 124, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, 120, -1, -1, -1, -1, + 292, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, 149, 290, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, -1, -1, -1, -1, -1, -1, + -1, -1, 288, 307, 205, 213, +}; + +unsigned short Parser_keys[] = { + 128, 227, 225, 225, 128, 228, 128, 244, + 128, 245, 128, 128, 128, 128, 45, 248, + 40, 249, 40, 249, 128, 250, 123, 128, + 123, 123, 123, 123, 128, 128, 123, 123, + 59, 128, 45, 248, 132, 132, 40, 292, + 40, 242, 40, 242, 59, 59, 128, 187, + 40, 292, 125, 228, 61, 141, 40, 242, + 59, 59, 59, 59, 40, 40, 40, 289, + 40, 289, 40, 249, 125, 244, 33, 281, + 33, 281, 40, 289, 128, 295, 59, 59, + 40, 249, 42, 295, 42, 295, 42, 295, + 59, 59, 59, 59, 40, 292, 44, 59, + 38, 144, 33, 281, 33, 201, 33, 181, + 33, 271, 33, 201, 33, 281, 33, 281, + 33, 201, 33, 201, 182, 279, 182, 279, + 179, 280, 134, 134, 33, 281, 59, 59, + 44, 59, 33, 281, 41, 41, 128, 294, + 40, 292, 59, 59, 40, 249, 59, 59, + 40, 249, 59, 59, 40, 249, 41, 44, + 33, 281, 179, 283, 182, 284, 182, 284, + 33, 281, 33, 281, 33, 281, 33, 281, + 33, 281, 33, 281, 33, 281, 33, 281, + 33, 281, 128, 293, 40, 275, 33, 274, + 40, 274, 40, 274, 40, 274, 40, 274, + 40, 274, 40, 206, 40, 206, 40, 206, + 40, 206, 202, 206, 202, 206, 44, 276, + 45, 281, 33, 281, 44, 255, 128, 245, + 41, 142, 40, 292, 40, 292, 40, 292, + 179, 186, 182, 279, 182, 279, 182, 186, + 38, 144, 128, 294, 128, 274, 40, 242, + 132, 132, 132, 132, 40, 274, 128, 274, + 128, 274, 44, 125, 132, 276, 61, 61, + 59, 59, 40, 274, 124, 124, 128, 128, + 182, 284, 182, 284, 186, 186, 33, 181, + 44, 44, 41, 41, 41, 44, 40, 289, + 44, 44, 41, 44, 125, 125, 125, 276, + 43, 275, 40, 274, 125, 125, 41, 41, + 41, 41, 0, 0 +}; + +unsigned int Parser_offsets[] = { + 0, 100, 101, 202, 319, 437, 438, 439, + 643, 853, 1063, 1186, 1192, 1193, 1194, 1195, + 1196, 1266, 1470, 1471, 1724, 1927, 2130, 2131, + 2191, 2444, 2548, 2629, 2832, 2833, 2834, 2835, + 3085, 3335, 3545, 3665, 3914, 4163, 4413, 4581, + 4582, 4792, 5046, 5300, 5554, 5555, 5556, 5809, + 5825, 5932, 6181, 6350, 6499, 6738, 6907, 7156, + 7405, 7574, 7743, 7841, 7939, 8041, 8042, 8291, + 8292, 8308, 8557, 8558, 8725, 8978, 8979, 9189, + 9190, 9400, 9401, 9611, 9615, 9864, 9969, 10072, + 10175, 10424, 10673, 10922, 11171, 11420, 11669, 11918, + 12167, 12416, 12582, 12818, 13060, 13295, 13530, 13765, + 14000, 14235, 14402, 14569, 14736, 14903, 14908, 14913, + 15146, 15383, 15632, 15844, 15962, 16064, 16317, 16570, + 16823, 16831, 16929, 17027, 17032, 17139, 17306, 17453, + 17656, 17657, 17658, 17893, 18040, 18187, 18269, 18414, + 18415, 18416, 18651, 18652, 18653, 18756, 18859, 18860, + 19009, 19010, 19011, 19015, 19265, 19266, 19270, 19271, + 19423, 19656, 19891, 19892, 19893, 19894 +}; + +unsigned short Parser_targs[] = { + 1, 2, 3, 4, 5, 6, 7, 8, + 9, 10, 11, 12, 13, 14, 15, 16, + 17, 18, 18, 19, 20, 21, 22, 23, + 24, 25, 26, 27, 28, 29, 30, 31, + 32, 33, 34, 35, 36, 37, 38, 39, + 40, 41, 42, 43, 44, 45, 46, 47, + 48, 49, 50, 50, 51, 52, 53, 54, + 55, 56, 57, 58, 59, 60, 61, 62, + 63, 64, 65, 66, 67, 68, 69, 70, + 71, 72, 73, 74, 75, 76, 77, 78, + 79, 80, 81, 82, 83, 84, 85, 86, + 87, 88, 89, 90, 91, 92, 93, 94, + 95, 96, 97, 98, 99, 100, 101, 102, + 103, 104, 105, 106, 107, 108, 109, 110, + 111, 112, 113, 114, 115, 116, 117, 118, + 119, 120, 121, 122, 123, 124, 125, 126, + 127, 128, 129, 130, 131, 132, 133, 134, + 135, 136, 137, 138, 139, 140, 141, 142, + 143, 144, 145, 146, 147, 148, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149, 149, 149, 149, + 149, 149, 149, 149, 149 +}; + +unsigned int Parser_actInds[] = { + 0, 2, 4, 6, 8, 10, 12, 14, + 16, 18, 20, 22, 24, 26, 28, 30, + 32, 34, 36, 39, 41, 43, 45, 47, + 49, 51, 53, 55, 57, 59, 61, 63, + 65, 67, 69, 71, 73, 75, 77, 79, + 81, 83, 85, 87, 89, 91, 93, 95, + 97, 99, 101, 103, 106, 108, 110, 112, + 114, 116, 118, 120, 122, 124, 126, 128, + 130, 132, 134, 136, 138, 140, 142, 144, + 146, 148, 150, 152, 154, 156, 158, 160, + 162, 164, 166, 168, 170, 172, 174, 176, + 178, 180, 182, 184, 186, 188, 190, 192, + 194, 196, 198, 200, 202, 204, 206, 208, + 210, 213, 215, 217, 219, 221, 223, 225, + 227, 229, 231, 233, 235, 237, 239, 241, + 243, 245, 247, 249, 251, 253, 255, 257, + 259, 261, 263, 265, 267, 269, 271, 273, + 275, 277, 279, 281, 283, 285, 287, 289, + 291, 293, 295, 297, 299, 301, 303, 305, + 307, 309, 311, 313, 315, 317, 319, 321, + 323, 325, 327, 329, 331, 333, 335, 337, + 339, 341, 343, 345, 347, 349, 351, 353, + 355, 357, 359, 361, 363, 365, 367, 369, + 371, 373, 375, 377, 379, 381, 383, 385, + 387, 389, 391, 393, 395, 397, 399, 401, + 403, 405, 407, 409, 411, 413, 415, 417, + 419, 421, 423, 425, 427, 429, 431, 433, + 435, 437, 439, 441, 443, 445, 447, 449, + 451, 453, 455, 457, 459, 461, 463, 465, + 467, 469, 471, 473, 475, 477, 479, 481, + 483, 485, 487, 489, 491, 493, 495, 497, + 499, 501, 503, 505, 507, 509, 511, 513, + 515, 517, 519, 521, 523, 525, 527, 529, + 531, 533, 535, 537, 539, 541, 543, 545, + 547, 549, 551, 553, 555, 557, 559, 561, + 563, 565, 567, 569, 571, 573, 575, 577, + 579, 581, 583, 585, 587, 589, 591, 593, + 595, 597, 599, 601, 603, 605, 607, 609, + 611, 613, 615, 617, 619, 621, 623, 625, + 627, 629, 631, 633, 635, 637, 639, 641, + 643, 645, 647, 649, 651, 653, 655, 657, + 659, 661, 663, 665, 667, 669, 671, 673, + 675, 677, 679, 681, 683, 685, 687, 689, + 691, 693, 695, 697, 699, 701, 703, 705, + 707, 709, 711, 713, 715, 717, 719, 721, + 723, 725, 727, 729, 731, 733, 735, 737, + 739, 741, 743, 745, 747, 749, 751, 753, + 755, 757, 759, 761, 763, 765, 767, 769, + 771, 773, 775, 777, 779, 781, 783, 785, + 787, 789, 791, 793, 795, 797, 799, 801, + 803, 805, 807, 809, 811 +}; + +unsigned int Parser_actions[] = { + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 214, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 90, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 0, 1, 0, 1, 0, + 1, 0, 1, 302, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 1, + 0, 1, 0, 1, 0, 1, 0, 2, + 0, 7, 0, 10, 0, 15, 0, 18, + 0, 71, 0, 75, 0, 79, 0, 83, + 0, 86, 0, 87, 0, 90, 0, 95, + 0, 99, 0, 103, 0, 107, 0, 111, + 0, 115, 0, 119, 0, 123, 0, 127, + 0, 131, 0, 135, 0, 139, 0, 142, + 0, 146, 0, 151, 0, 155, 0, 159, + 0, 163, 0, 167, 0, 171, 0, 175, + 0, 179, 0, 182, 0, 186, 0, 190, + 0, 195, 0, 199, 0, 203, 0, 207, + 0, 211, 0, 214, 0, 219, 0, 223, + 0, 227, 0, 231, 0, 235, 0, 239, + 0, 243, 0, 246, 0, 251, 0, 254, + 0, 259, 0, 263, 0, 267, 0, 271, + 0, 275, 0, 279, 0, 283, 0, 287, + 0, 291, 0, 295, 0, 299, 0, 302, + 0, 306, 0, 310, 0, 314, 0, 318, + 0, 323, 0, 327, 0, 331, 0, 335, + 0, 339, 0, 343, 0, 347, 0, 351, + 0, 355, 0, 359, 0, 363, 0, 367, + 0, 371, 0, 375, 0, 379, 0, 383, + 0, 387, 0, 391, 0, 395, 0, 399, + 0, 403, 0, 407, 0, 411, 0, 415, + 0, 419, 0, 423, 0, 427, 0, 431, + 0, 435, 0, 439, 0, 443, 0, 447, + 0, 451, 0, 455, 0, 459, 0, 463, + 0, 467, 0, 471, 0, 475, 0, 479, + 0, 483, 0, 487, 0, 491, 0, 495, + 0, 499, 0, 503, 0, 507, 0, 511, + 0, 515, 0, 519, 0, 523, 0, 527, + 0, 531, 0, 535, 0, 539, 0, 543, + 0, 547, 0, 551, 0, 555, 0, 559, + 0, 563, 0, 567, 0, 571, 0, 575, + 0, 579, 0, 583, 0, 587, 0, 591, + 0, 595, 0, 599, 0, 603, 0, 607, + 0, 610, 0, 611, 0, 615, 0, 618, + 0, 623, 0, 627, 0, 631, 0, 635, + 0, 638, 0, 643, 0, 647, 0, 651, + 0, 655, 0, 659, 0, 663, 0, 667, + 0, 671, 0, 675, 0, 679, 0, 683, + 0, 687, 0, 691, 0, 694, 0, 698, + 0, 702, 0, 703, 0, 707, 0, 711, + 0, 715, 0, 719, 0, 723, 0, 726, + 0, 727, 0, 730, 0, 731, 0, 735, + 0, 739, 0, 743, 0, 747, 0, 750, + 0, 755, 0, 758, 0, 763, 0, 767, + 0, 771, 0, 775, 0, 779, 0, 782, + 0, 786, 0, 791, 0, 795, 0, 798, + 0, 803, 0, 807, 0, 811, 0, 815, + 0, 819, 0, 823, 0, 827, 0, 831, + 0, 835, 0, 839, 0, 843, 0, 847, + 0, 851, 0, 855, 0, 859, 0, 863, + 0, 867, 0, 871, 0, 875, 0, 879, + 0, 883, 0, 886, 0, 891, 0, 895, + 0, 899, 0, 903, 0, 907, 0, 911, + 0, 915, 0, 919, 0, 923, 0, 927, + 0, 931, 0, 935, 0, 939, 0, 943, + 0, 947, 0, 951, 0, 955, 0, 959, + 0, 963, 0, 967, 0, 970, 0, 974, + 0, 978, 0, 983, 0, 986, 0, 991, + 0, 995, 0, 23, 0, 27, 0, 31, + 0, 35, 0, 39, 0, 43, 0, 47, + 0, 51, 0, 55, 0, 59, 0, 63, + 0, 67, 0, 1, 0 +}; + +int Parser_commitLen[] = { + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, + 1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, 2 +}; + +char Parser_prodLengths[] = { + 1, 3, 0, 2, 0, 1, 1, 1, + 1, 1, 1, 1, 1, 1, 1, 1, + 1, 3, 4, 4, 1, 1, 0, 4, + 5, 5, 1, 5, 4, 3, 4, 3, + 3, 5, 2, 0, 1, 4, 2, 1, + 1, 1, 3, 2, 1, 0, 3, 1, + 3, 3, 3, 3, 1, 1, 2, 3, + 3, 3, 3, 1, 3, 1, 3, 1, + 3, 3, 7, 3, 4, 3, 3, 3, + 3, 3, 7, 1, 1, 1, 1, 1, + 1, 2, 1, 2, 1, 2, 1, 1, + 1, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 2, 1, + 2, 1, 2, 1, 2, 1, 3, 1, + 1, 3, 1, 1, 1, 2, 2, 1, + 2, 2, 2, 2, 4, 5, 5, 6, + 1, 1, 2, 2, 1, 1, 1, 1, + 3, 3, 3, 3, 3, 1, 1, 1, + 2, 1, 2, 0, 2, 1, 3, 3, + 1, 1, 2, 0, 1, 3, 2, 0, + 1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 2, 3, 3, 4, 3, 4, + 3, 4, 2, 2, 2, 0, 1, 1, + 1, 1, 1, 1, 1, 1, 1, 1, + 1, 1, 1, 1, 1, 1, 1, 1, + 1, 4, 2, 0, 2, 1, 0, 3, + 1, 1 +}; + +unsigned short Parser_prodLhsIds[] = { + 227, 226, 226, 228, 228, 229, 229, 229, + 229, 229, 229, 229, 229, 229, 229, 229, + 229, 241, 239, 240, 243, 244, 244, 238, + 230, 231, 245, 232, 233, 233, 234, 235, + 236, 237, 250, 250, 247, 247, 251, 251, + 252, 252, 252, 253, 253, 253, 246, 246, + 256, 256, 256, 256, 256, 257, 258, 258, + 258, 258, 258, 258, 259, 259, 260, 260, + 262, 262, 262, 262, 262, 262, 262, 262, + 262, 262, 262, 262, 263, 263, 263, 263, + 266, 266, 266, 266, 266, 266, 266, 266, + 266, 267, 267, 267, 267, 267, 267, 267, + 267, 267, 267, 267, 267, 268, 268, 268, + 268, 268, 268, 268, 268, 268, 268, 268, + 268, 269, 269, 269, 269, 269, 269, 269, + 269, 269, 269, 269, 269, 270, 270, 270, + 270, 270, 270, 270, 270, 270, 270, 270, + 270, 271, 271, 271, 271, 271, 271, 271, + 271, 271, 271, 271, 271, 254, 254, 254, + 274, 255, 265, 264, 275, 275, 275, 272, + 273, 273, 273, 273, 273, 273, 273, 273, + 273, 276, 277, 277, 277, 278, 278, 278, + 278, 278, 278, 278, 278, 281, 281, 248, + 248, 248, 280, 280, 282, 282, 283, 283, + 283, 283, 279, 279, 284, 284, 242, 242, + 285, 285, 285, 288, 288, 288, 288, 288, + 288, 286, 286, 286, 286, 286, 286, 286, + 286, 286, 286, 286, 249, 249, 291, 291, + 291, 287, 287, 287, 287, 287, 287, 287, + 292, 292, 292, 292, 292, 289, 289, 289, + 289, 289, 261, 293, 290, 295, 295, 294, + 294, 296 +}; + +const char *Parser_prodNames[] = { + "start-1", + "section_list-1", + "section_list-2", + "statement_list-1", + "statement_list-2", + "statement-1", + "statement-2", + "statement-3", + "statement-4", + "statement-5", + "statement-6", + "statement-7", + "statement-8", + "statement-9", + "statement-10", + "statement-11", + "statement-12", + "length_spec-1", + "pre_push_spec-1", + "post_pop_spec-1", + "export_open-1", + "opt_export-1", + "opt_export-2", + "export_block-1", + "assignment-1", + "instantiation-1", + "machine_name-1", + "action_spec-1", + "alphtype_spec-1", + "alphtype_spec-2", + "range_spec-1", + "getkey_spec-1", + "access_spec-1", + "variable_spec-1", + "opt_whitespace-1", + "opt_whitespace-2", + "join_or_lm-1", + "join_or_lm-2", + "lm_part_list-1", + "lm_part_list-2", + "longest_match_part-1", + "longest_match_part-2", + "longest_match_part-3", + "opt_lm_part_action-1", + "opt_lm_part_action-2", + "opt_lm_part_action-3", + "join-1", + "join-2", + "expression-1", + "expression-2", + "expression-3", + "expression-4", + "expression-5", + "term_short-1", + "term-1", + "term-2", + "term-3", + "term-4", + "term-5", + "term-6", + "factor_with_label-1", + "factor_with_label-2", + "factor_with_ep-1", + "factor_with_ep-2", + "factor_with_aug-1", + "factor_with_aug-2", + "factor_with_aug-3", + "factor_with_aug-4", + "factor_with_aug-5", + "factor_with_aug-6", + "factor_with_aug-7", + "factor_with_aug-8", + "factor_with_aug-9", + "factor_with_aug-10", + "factor_with_aug-11", + "factor_with_aug-12", + "aug_type_base-1", + "aug_type_base-2", + "aug_type_base-3", + "aug_type_base-4", + "aug_type_cond-1", + "aug_type_cond-2", + "aug_type_cond-3", + "aug_type_cond-4", + "aug_type_cond-5", + "aug_type_cond-6", + "aug_type_cond-7", + "aug_type_cond-8", + "aug_type_cond-9", + "aug_type_to_state-1", + "aug_type_to_state-2", + "aug_type_to_state-3", + "aug_type_to_state-4", + "aug_type_to_state-5", + "aug_type_to_state-6", + "aug_type_to_state-7", + "aug_type_to_state-8", + "aug_type_to_state-9", + "aug_type_to_state-10", + "aug_type_to_state-11", + "aug_type_to_state-12", + "aug_type_from_state-1", + "aug_type_from_state-2", + "aug_type_from_state-3", + "aug_type_from_state-4", + "aug_type_from_state-5", + "aug_type_from_state-6", + "aug_type_from_state-7", + "aug_type_from_state-8", + "aug_type_from_state-9", + "aug_type_from_state-10", + "aug_type_from_state-11", + "aug_type_from_state-12", + "aug_type_eof-1", + "aug_type_eof-2", + "aug_type_eof-3", + "aug_type_eof-4", + "aug_type_eof-5", + "aug_type_eof-6", + "aug_type_eof-7", + "aug_type_eof-8", + "aug_type_eof-9", + "aug_type_eof-10", + "aug_type_eof-11", + "aug_type_eof-12", + "aug_type_gbl_error-1", + "aug_type_gbl_error-2", + "aug_type_gbl_error-3", + "aug_type_gbl_error-4", + "aug_type_gbl_error-5", + "aug_type_gbl_error-6", + "aug_type_gbl_error-7", + "aug_type_gbl_error-8", + "aug_type_gbl_error-9", + "aug_type_gbl_error-10", + "aug_type_gbl_error-11", + "aug_type_gbl_error-12", + "aug_type_local_error-1", + "aug_type_local_error-2", + "aug_type_local_error-3", + "aug_type_local_error-4", + "aug_type_local_error-5", + "aug_type_local_error-6", + "aug_type_local_error-7", + "aug_type_local_error-8", + "aug_type_local_error-9", + "aug_type_local_error-10", + "aug_type_local_error-11", + "aug_type_local_error-12", + "action_embed-1", + "action_embed-2", + "action_embed-3", + "action_embed_word-1", + "action_embed_block-1", + "priority_name-1", + "priority_aug-1", + "priority_aug_num-1", + "priority_aug_num-2", + "priority_aug_num-3", + "local_err_name-1", + "factor_with_rep-1", + "factor_with_rep-2", + "factor_with_rep-3", + "factor_with_rep-4", + "factor_with_rep-5", + "factor_with_rep-6", + "factor_with_rep-7", + "factor_with_rep-8", + "factor_with_rep-9", + "factor_rep_num-1", + "factor_with_neg-1", + "factor_with_neg-2", + "factor_with_neg-3", + "factor-1", + "factor-2", + "factor-3", + "factor-4", + "factor-5", + "factor-6", + "factor-7", + "factor-8", + "range_lit-1", + "range_lit-2", + "alphabet_num-1", + "alphabet_num-2", + "alphabet_num-3", + "regular_expr-1", + "regular_expr-2", + "regular_expr_item-1", + "regular_expr_item-2", + "regular_expr_char-1", + "regular_expr_char-2", + "regular_expr_char-3", + "regular_expr_char-4", + "regular_expr_or_data-1", + "regular_expr_or_data-2", + "regular_expr_or_char-1", + "regular_expr_or_char-2", + "inline_block-1", + "inline_block-2", + "inline_block_item-1", + "inline_block_item-2", + "inline_block_item-3", + "inline_block_symbol-1", + "inline_block_symbol-2", + "inline_block_symbol-3", + "inline_block_symbol-4", + "inline_block_symbol-5", + "inline_block_symbol-6", + "inline_block_interpret-1", + "inline_block_interpret-2", + "inline_block_interpret-3", + "inline_block_interpret-4", + "inline_block_interpret-5", + "inline_block_interpret-6", + "inline_block_interpret-7", + "inline_block_interpret-8", + "inline_block_interpret-9", + "inline_block_interpret-10", + "inline_block_interpret-11", + "inline_expr-1", + "inline_expr-2", + "inline_expr_item-1", + "inline_expr_item-2", + "inline_expr_item-3", + "inline_expr_any-1", + "inline_expr_any-2", + "inline_expr_any-3", + "inline_expr_any-4", + "inline_expr_any-5", + "inline_expr_any-6", + "inline_expr_any-7", + "inline_expr_symbol-1", + "inline_expr_symbol-2", + "inline_expr_symbol-3", + "inline_expr_symbol-4", + "inline_expr_symbol-5", + "inline_expr_interpret-1", + "inline_expr_interpret-2", + "inline_expr_interpret-3", + "inline_expr_interpret-4", + "inline_expr_interpret-5", + "local_state_ref-1", + "no_name_sep-1", + "state_ref-1", + "opt_name_sep-1", + "opt_name_sep-2", + "state_ref_names-1", + "state_ref_names-2", + "_start-1" +}; + +const char *Parser_lelNames[] = { + "D-0", + "D-1", + "D-2", + "D-3", + "D-4", + "D-5", + "D-6", + "D-7", + "D-8", + "D-9", + "D-10", + "D-11", + "D-12", + "D-13", + "D-14", + "D-15", + "D-16", + "D-17", + "D-18", + "D-19", + "D-20", + "D-21", + "D-22", + "D-23", + "D-24", + "D-25", + "D-26", + "D-27", + "D-28", + "D-29", + "D-30", + "D-31", + "D-32", + "!", + "\"", + "#", + "$", + "%", + "&", + "'", + "(", + ")", + "*", + "+", + ",", + "-", + ".", + "/", + "0", + "1", + "2", + "3", + "4", + "5", + "6", + "7", + "8", + "9", + ":", + ";", + "<", + "=", + ">", + "?", + "@", + "A", + "B", + "C", + "D", + "E", + "F", + "G", + "H", + "I", + "J", + "K", + "L", + "M", + "N", + "O", + "P", + "Q", + "R", + "S", + "T", + "U", + "V", + "W", + "X", + "Y", + "Z", + "[", + "\\", + "]", + "^", + "_", + "`", + "a", + "b", + "c", + "d", + "e", + "f", + "g", + "h", + "i", + "j", + "k", + "l", + "m", + "n", + "o", + "p", + "q", + "r", + "s", + "t", + "u", + "v", + "w", + "x", + "y", + "z", + "{", + "|", + "}", + "~", + "D-127", + "TK_Word", + "TK_Literal", + "TK_Number", + "TK_EndSection", + "TK_UInt", + "TK_Hex", + "TK_DotDot", + "TK_ColonGt", + "TK_ColonGtGt", + "TK_LtColon", + "TK_Arrow", + "TK_DoubleArrow", + "TK_StarStar", + "TK_ColonEquals", + "TK_NameSep", + "TK_BarStar", + "TK_DashDash", + "TK_StartCond", + "TK_AllCond", + "TK_LeavingCond", + "TK_Middle", + "TK_StartGblError", + "TK_AllGblError", + "TK_FinalGblError", + "TK_NotFinalGblError", + "TK_NotStartGblError", + "TK_MiddleGblError", + "TK_StartLocalError", + "TK_AllLocalError", + "TK_FinalLocalError", + "TK_NotFinalLocalError", + "TK_NotStartLocalError", + "TK_MiddleLocalError", + "TK_StartEOF", + "TK_AllEOF", + "TK_FinalEOF", + "TK_NotFinalEOF", + "TK_NotStartEOF", + "TK_MiddleEOF", + "TK_StartToState", + "TK_AllToState", + "TK_FinalToState", + "TK_NotFinalToState", + "TK_NotStartToState", + "TK_MiddleToState", + "TK_StartFromState", + "TK_AllFromState", + "TK_FinalFromState", + "TK_NotFinalFromState", + "TK_NotStartFromState", + "TK_MiddleFromState", + "RE_Slash", + "RE_SqOpen", + "RE_SqOpenNeg", + "RE_SqClose", + "RE_Dot", + "RE_Star", + "RE_Dash", + "RE_Char", + "IL_WhiteSpace", + "IL_Comment", + "IL_Literal", + "IL_Symbol", + "KW_Machine", + "KW_Include", + "KW_Import", + "KW_Write", + "KW_Action", + "KW_AlphType", + "KW_Range", + "KW_GetKey", + "KW_InWhen", + "KW_When", + "KW_OutWhen", + "KW_Eof", + "KW_Err", + "KW_Lerr", + "KW_To", + "KW_From", + "KW_Export", + "KW_PrePush", + "KW_PostPop", + "KW_Length", + "KW_Break", + "KW_Exec", + "KW_Hold", + "KW_PChar", + "KW_Char", + "KW_Goto", + "KW_Call", + "KW_Ret", + "KW_CurState", + "KW_TargState", + "KW_Entry", + "KW_Next", + "KW_Variable", + "KW_Access", + "Parser_tk_eof", + "section_list", + "start", + "statement_list", + "statement", + "assignment", + "instantiation", + "action_spec", + "alphtype_spec", + "range_spec", + "getkey_spec", + "access_spec", + "variable_spec", + "export_block", + "pre_push_spec", + "post_pop_spec", + "length_spec", + "inline_block", + "export_open", + "opt_export", + "machine_name", + "join", + "join_or_lm", + "alphabet_num", + "inline_expr", + "opt_whitespace", + "lm_part_list", + "longest_match_part", + "opt_lm_part_action", + "action_embed", + "action_embed_block", + "expression", + "term_short", + "term", + "factor_with_label", + "factor_with_ep", + "local_state_ref", + "factor_with_aug", + "aug_type_base", + "priority_aug", + "priority_name", + "aug_type_cond", + "aug_type_to_state", + "aug_type_from_state", + "aug_type_eof", + "aug_type_gbl_error", + "aug_type_local_error", + "local_err_name", + "factor_with_rep", + "action_embed_word", + "priority_aug_num", + "factor_rep_num", + "factor_with_neg", + "factor", + "regular_expr_or_data", + "regular_expr", + "range_lit", + "regular_expr_item", + "regular_expr_char", + "regular_expr_or_char", + "inline_block_item", + "inline_block_interpret", + "inline_expr_any", + "inline_block_symbol", + "inline_expr_interpret", + "state_ref", + "inline_expr_item", + "inline_expr_symbol", + "no_name_sep", + "state_ref_names", + "opt_name_sep", + "_start" +}; + +#line 1449 "rlparse.kl" + + +void Parser::init() +{ + #line 3855 "rlparse.cpp" + curs = Parser_startState; + pool = 0; + block = (struct Parser_Block*) malloc( sizeof(struct Parser_Block) ); + block->next = 0; + freshEl = block->data; + #ifdef KELBT_LOG_ACTIONS + cerr << "allocating 8128 LangEls" << endl; + #endif + stackTop = freshEl; + stackTop->type = 0; + stackTop->state = -1; + stackTop->next = 0; + stackTop->child = 0; + stackTop->causeReduce = 0; + freshPos = 1; + lastFinal = stackTop; + numRetry = 0; + numNodes = 0; + errCount = 0; +#line 1454 "rlparse.kl" +} + +int Parser::parseLangEl( int type, const Token *token ) +{ + #line 3880 "rlparse.cpp" +#define reject() induceReject = 1 + + int pos, targState; + unsigned int *action; + int rhsLen; + struct Parser_LangEl *rhs[32]; + struct Parser_LangEl *lel = 0; + struct Parser_LangEl *input = 0; + struct Parser_LangEl *queue = 0; + char induceReject; + + if ( curs < 0 ) + return 0; + + if ( pool == 0 ) { + if ( freshPos == 8128 ) { + struct Parser_Block* newBlock = (struct Parser_Block*) malloc( sizeof(struct Parser_Block) ); + newBlock->next = block; + block = newBlock; + freshEl = newBlock->data; + #ifdef KELBT_LOG_ACTIONS + cerr << "allocating 8128 LangEls" << endl; + #endif + freshPos = 0; + } + queue = freshEl + freshPos++; + } + else { + queue = pool; + pool = pool->next; + } + numNodes += 1; + + queue->type = type; + queue->user.token = *token; + queue->next = 0; + queue->retry = 0; + queue->child = 0; + queue->causeReduce = 0; + +again: + if ( input == 0 ) { + if ( queue == 0 ) + goto _out; + + input = queue; + queue = queue->next; + input->next = 0; + } + + lel = input; + if ( lel->type < Parser_keys[curs<<1] || lel->type > Parser_keys[(curs<<1)+1] ) + goto parseError; + + pos = Parser_indicies[Parser_offsets[curs] + (lel->type - Parser_keys[curs<<1])]; + if ( pos < 0 ) + goto parseError; + + induceReject = 0; + targState = Parser_targs[pos]; + action = Parser_actions + Parser_actInds[pos]; + if ( lel->retry & 0x0000ffff ) + action += (lel->retry & 0x0000ffff); + + if ( *action & 0x1 ) { + #ifdef KELBT_LOG_ACTIONS + cerr << "shifted: " << Parser_lelNames[lel->type]; + #endif + input = input->next; + lel->state = curs; + lel->next = stackTop; + stackTop = lel; + + if ( action[1] == 0 ) + lel->retry &= 0xffff0000; + else { + lel->retry += 1; + numRetry += 1; + #ifdef KELBT_LOG_ACTIONS + cerr << " retry: " << stackTop; + #endif + } + #ifdef KELBT_LOG_ACTIONS + cerr << endl; + #endif + } + + if ( Parser_commitLen[pos] != 0 ) { + struct Parser_LangEl *commitHead = stackTop, *lel; + int sp = 0, doExec = 0; + #ifdef KELBT_LOG_ACTIONS + cerr << "commit encountered, executing final actions" << endl; + #endif + if ( Parser_commitLen[pos] < 0 ) + commitHead = commitHead->next; + + lel = commitHead; + +commit_head: + if ( lel == lastFinal ) { + doExec = 1; + goto commit_base; + } + + if ( lel->next != 0 ) { + sp += 1; + lel->next->prev = lel; + lel = lel->next; + lel->retry = 0; + goto commit_head; + } + +commit_reverse: + + if ( lel->child != 0 ) { + sp += 1; + lel->child->prev = lel; + lel = lel->child; + lel->retry = 1; + goto commit_head; + } + +commit_upwards: + + if ( doExec ) { + if ( lel->type < 226 ) { + } + else { + struct Parser_LangEl *redLel = lel; + if ( redLel->child != 0 ) { + int r = Parser_prodLengths[redLel->reduction] - 1; + struct Parser_LangEl *rhsEl = redLel->child; + while ( rhsEl != 0 ) { + rhs[r--] = rhsEl; + rhsEl = rhsEl->next; + } + } +switch ( lel->reduction ) { +case 17: { +Token *__ref0 = (Token*)&rhs[1]->user.token; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Token *__ref2 = (Token*)&rhs[1]->user.token; +#line 61 "rlparse.kl" + + LengthDef *lengthDef = new LengthDef( (__ref0)->data ); + pd->lengthDefList.append( lengthDef ); + + /* Generic creation of machine for instantiation and assignment. */ + MachineDef *machineDef = new MachineDef( lengthDef ); + tryMachineDef( (__ref1)->loc, (__ref2)->data, machineDef, false ); + + +#line 4031 "rlparse.cpp" +} break; +case 18: { +Token *__ref0 = (Token*)&rhs[1]->user.token; +Parser_Lel_inline_list *__ref1 = (Parser_Lel_inline_list*)&rhs[2]->user.inline_list; +#line 72 "rlparse.kl" + + if ( pd->prePushExpr != 0 ) { + /* Recover by just ignoring the duplicate. */ + error((__ref0)->loc) << "pre_push code already defined" << endl; + } + + pd->prePushExpr = (__ref1)->inlineList; + + +#line 4046 "rlparse.cpp" +} break; +case 19: { +Token *__ref0 = (Token*)&rhs[1]->user.token; +Parser_Lel_inline_list *__ref1 = (Parser_Lel_inline_list*)&rhs[2]->user.inline_list; +#line 84 "rlparse.kl" + + if ( pd->postPopExpr != 0 ) { + /* Recover by just ignoring the duplicate. */ + error((__ref0)->loc) << "post_pop code already defined" << endl; + } + + pd->postPopExpr = (__ref1)->inlineList; + + +#line 4061 "rlparse.cpp" +} break; +case 20: { +#line 95 "rlparse.kl" + + exportContext.append( true ); + + +#line 4069 "rlparse.cpp" +} break; +case 21: { +Parser_Lel_opt_export *__ref0 = (Parser_Lel_opt_export*)&redLel->user.opt_export; +#line 104 "rlparse.kl" + (__ref0)->isSet = true; + +#line 4076 "rlparse.cpp" +} break; +case 22: { +Parser_Lel_opt_export *__ref0 = (Parser_Lel_opt_export*)&redLel->user.opt_export; +#line 105 "rlparse.kl" + (__ref0)->isSet = false; + +#line 4083 "rlparse.cpp" +} break; +case 23: { +#line 108 "rlparse.kl" + + exportContext.remove( exportContext.length()-1 ); + + +#line 4091 "rlparse.cpp" +} break; +case 24: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&rhs[1]->user.token_type; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&rhs[1]->user.token_type; +Parser_Lel_join *__ref2 = (Parser_Lel_join*)&rhs[3]->user.join; +Parser_Lel_token_type *__ref3 = (Parser_Lel_token_type*)&rhs[1]->user.token_type; +Parser_Lel_token_type *__ref4 = (Parser_Lel_token_type*)&rhs[1]->user.token_type; +Parser_Lel_opt_export *__ref5 = (Parser_Lel_opt_export*)&rhs[0]->user.opt_export; +Parser_Lel_join *__ref6 = (Parser_Lel_join*)&rhs[3]->user.join; +Token *__ref7 = (Token*)&rhs[2]->user.token; +#line 113 "rlparse.kl" + + /* Main machine must be an instance. */ + bool isInstance = false; + if ( strcmp((__ref0)->token.data, mainMachine) == 0 ) { + warning((__ref1)->token.loc) << + "main machine will be implicitly instantiated" << endl; + isInstance = true; + } + + /* Generic creation of machine for instantiation and assignment. */ + MachineDef *machineDef = new MachineDef( (__ref2)->join ); + tryMachineDef( (__ref3)->token.loc, (__ref4)->token.data, machineDef, isInstance ); + + if ( (__ref5)->isSet ) + exportContext.remove( exportContext.length()-1 ); + + (__ref6)->join->loc = (__ref7)->loc; + + +#line 4122 "rlparse.cpp" +} break; +case 25: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&rhs[1]->user.token_type; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&rhs[1]->user.token_type; +Parser_Lel_join_or_lm *__ref2 = (Parser_Lel_join_or_lm*)&rhs[3]->user.join_or_lm; +Parser_Lel_opt_export *__ref3 = (Parser_Lel_opt_export*)&rhs[0]->user.opt_export; +Parser_Lel_join_or_lm *__ref4 = (Parser_Lel_join_or_lm*)&rhs[3]->user.join_or_lm; +Parser_Lel_join_or_lm *__ref5 = (Parser_Lel_join_or_lm*)&rhs[3]->user.join_or_lm; +Token *__ref6 = (Token*)&rhs[2]->user.token; +#line 133 "rlparse.kl" + + /* Generic creation of machine for instantiation and assignment. */ + tryMachineDef( (__ref0)->token.loc, (__ref1)->token.data, (__ref2)->machineDef, true ); + + if ( (__ref3)->isSet ) + exportContext.remove( exportContext.length()-1 ); + + /* Pass a location to join_or_lm */ + if ( (__ref4)->machineDef->join != 0 ) + (__ref5)->machineDef->join->loc = (__ref6)->loc; + + +#line 4145 "rlparse.cpp" +} break; +case 26: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_token_type *__ref2 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref3 = (Token*)&rhs[0]->user.token; +#line 153 "rlparse.kl" + + /* Make/get the priority key. The name may have already been referenced + * and therefore exist. */ + PriorDictEl *priorDictEl; + if ( pd->priorDict.insert( (__ref0)->data, pd->nextPriorKey, &priorDictEl ) ) + pd->nextPriorKey += 1; + pd->curDefPriorKey = priorDictEl->value; + + /* Make/get the local error key. */ + LocalErrDictEl *localErrDictEl; + if ( pd->localErrDict.insert( (__ref1)->data, pd->nextLocalErrKey, &localErrDictEl ) ) + pd->nextLocalErrKey += 1; + pd->curDefLocalErrKey = localErrDictEl->value; + + (__ref2)->token = *(__ref3); + + +#line 4170 "rlparse.cpp" +} break; +case 27: { +Token *__ref0 = (Token*)&rhs[1]->user.token; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Token *__ref2 = (Token*)&rhs[1]->user.token; +Token *__ref3 = (Token*)&rhs[2]->user.token; +Token *__ref4 = (Token*)&rhs[1]->user.token; +Parser_Lel_inline_list *__ref5 = (Parser_Lel_inline_list*)&rhs[3]->user.inline_list; +#line 171 "rlparse.kl" + + if ( pd->actionDict.find( (__ref0)->data ) ) { + /* Recover by just ignoring the duplicate. */ + error((__ref1)->loc) << "action \"" << (__ref2)->data << "\" already defined" << endl; + } + else { + //cerr << "NEW ACTION " << $2->data << " " << $4->inlineList << endl; + /* Add the action to the list of actions. */ + Action *newAction = new Action( (__ref3)->loc, (__ref4)->data, + (__ref5)->inlineList, pd->nextCondId++ ); + + /* Insert to list and dict. */ + pd->actionList.append( newAction ); + pd->actionDict.insert( newAction ); + } + + +#line 4197 "rlparse.cpp" +} break; +case 28: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Token *__ref2 = (Token*)&rhs[2]->user.token; +Token *__ref3 = (Token*)&rhs[1]->user.token; +Token *__ref4 = (Token*)&rhs[1]->user.token; +Token *__ref5 = (Token*)&rhs[2]->user.token; +#line 191 "rlparse.kl" + + if ( ! pd->setAlphType( (__ref0)->loc, (__ref1)->data, (__ref2)->data ) ) { + // Recover by ignoring the alphtype statement. + error((__ref3)->loc) << "\"" << (__ref4)->data << + " " << (__ref5)->data << "\" is not a valid alphabet type" << endl; + } + + +#line 4215 "rlparse.cpp" +} break; +case 29: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Token *__ref2 = (Token*)&rhs[1]->user.token; +Token *__ref3 = (Token*)&rhs[1]->user.token; +#line 200 "rlparse.kl" + + if ( ! pd->setAlphType( (__ref0)->loc, (__ref1)->data ) ) { + // Recover by ignoring the alphtype statement. + error((__ref2)->loc) << "\"" << (__ref3)->data << + "\" is not a valid alphabet type" << endl; + } + + +#line 4231 "rlparse.cpp" +} break; +case 30: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&rhs[1]->user.token_type; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&rhs[2]->user.token_type; +Parser_Lel_token_type *__ref2 = (Parser_Lel_token_type*)&rhs[1]->user.token_type; +Parser_Lel_token_type *__ref3 = (Parser_Lel_token_type*)&rhs[2]->user.token_type; +#line 210 "rlparse.kl" + + // Save the upper and lower ends of the range and emit the line number. + pd->lowerNum = (__ref0)->token.data; + pd->upperNum = (__ref1)->token.data; + pd->rangeLowLoc = (__ref2)->token.loc; + pd->rangeHighLoc = (__ref3)->token.loc; + + +#line 4247 "rlparse.cpp" +} break; +case 31: { +Parser_Lel_inline_list *__ref0 = (Parser_Lel_inline_list*)&rhs[1]->user.inline_list; +#line 219 "rlparse.kl" + + pd->getKeyExpr = (__ref0)->inlineList; + + +#line 4256 "rlparse.cpp" +} break; +case 32: { +Parser_Lel_inline_list *__ref0 = (Parser_Lel_inline_list*)&rhs[1]->user.inline_list; +#line 224 "rlparse.kl" + + pd->accessExpr = (__ref0)->inlineList; + + +#line 4265 "rlparse.cpp" +} break; +case 33: { +Token *__ref0 = (Token*)&rhs[2]->user.token; +Parser_Lel_inline_list *__ref1 = (Parser_Lel_inline_list*)&rhs[3]->user.inline_list; +Token *__ref2 = (Token*)&rhs[2]->user.token; +#line 229 "rlparse.kl" + + /* FIXME: Need to implement the rest of this. */ + bool wasSet = pd->setVariable( (__ref0)->data, (__ref1)->inlineList ); + if ( !wasSet ) + error((__ref2)->loc) << "bad variable name" << endl; + + +#line 4279 "rlparse.cpp" +} break; +case 36: { +Parser_Lel_join_or_lm *__ref0 = (Parser_Lel_join_or_lm*)&redLel->user.join_or_lm; +Parser_Lel_join *__ref1 = (Parser_Lel_join*)&rhs[0]->user.join; +#line 249 "rlparse.kl" + + (__ref0)->machineDef = new MachineDef( (__ref1)->join ); + + +#line 4289 "rlparse.cpp" +} break; +case 37: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +Parser_Lel_lm_part_list *__ref1 = (Parser_Lel_lm_part_list*)&rhs[1]->user.lm_part_list; +Parser_Lel_lm_part_list *__ref2 = (Parser_Lel_lm_part_list*)&rhs[1]->user.lm_part_list; +Parser_Lel_join_or_lm *__ref3 = (Parser_Lel_join_or_lm*)&redLel->user.join_or_lm; +#line 253 "rlparse.kl" + + /* Create a new factor going to a longest match structure. Record + * in the parse data that we have a longest match. */ + LongestMatch *lm = new LongestMatch( (__ref0)->loc, (__ref1)->lmPartList ); + pd->lmList.append( lm ); + for ( LmPartList::Iter lmp = *((__ref2)->lmPartList); lmp.lte(); lmp++ ) + lmp->longestMatch = lm; + (__ref3)->machineDef = new MachineDef( lm ); + + +#line 4307 "rlparse.cpp" +} break; +case 38: { +Parser_Lel_longest_match_part *__ref0 = (Parser_Lel_longest_match_part*)&rhs[1]->user.longest_match_part; +Parser_Lel_lm_part_list *__ref1 = (Parser_Lel_lm_part_list*)&rhs[0]->user.lm_part_list; +Parser_Lel_longest_match_part *__ref2 = (Parser_Lel_longest_match_part*)&rhs[1]->user.longest_match_part; +Parser_Lel_lm_part_list *__ref3 = (Parser_Lel_lm_part_list*)&redLel->user.lm_part_list; +Parser_Lel_lm_part_list *__ref4 = (Parser_Lel_lm_part_list*)&rhs[0]->user.lm_part_list; +#line 270 "rlparse.kl" + + if ( (__ref0)->lmPart != 0 ) + (__ref1)->lmPartList->append( (__ref2)->lmPart ); + (__ref3)->lmPartList = (__ref4)->lmPartList; + + +#line 4322 "rlparse.cpp" +} break; +case 39: { +Parser_Lel_lm_part_list *__ref0 = (Parser_Lel_lm_part_list*)&redLel->user.lm_part_list; +Parser_Lel_longest_match_part *__ref1 = (Parser_Lel_longest_match_part*)&rhs[0]->user.longest_match_part; +Parser_Lel_lm_part_list *__ref2 = (Parser_Lel_lm_part_list*)&redLel->user.lm_part_list; +Parser_Lel_longest_match_part *__ref3 = (Parser_Lel_longest_match_part*)&rhs[0]->user.longest_match_part; +#line 277 "rlparse.kl" + + /* Create a new list with the part. */ + (__ref0)->lmPartList = new LmPartList; + if ( (__ref1)->lmPart != 0 ) + (__ref2)->lmPartList->append( (__ref3)->lmPart ); + + +#line 4337 "rlparse.cpp" +} break; +case 40: { +Parser_Lel_longest_match_part *__ref0 = (Parser_Lel_longest_match_part*)&redLel->user.longest_match_part; +#line 290 "rlparse.kl" + (__ref0)->lmPart = 0; + +#line 4344 "rlparse.cpp" +} break; +case 41: { +Parser_Lel_longest_match_part *__ref0 = (Parser_Lel_longest_match_part*)&redLel->user.longest_match_part; +#line 292 "rlparse.kl" + (__ref0)->lmPart = 0; + +#line 4351 "rlparse.cpp" +} break; +case 42: { +Parser_Lel_longest_match_part *__ref0 = (Parser_Lel_longest_match_part*)&redLel->user.longest_match_part; +Parser_Lel_opt_lm_part_action *__ref1 = (Parser_Lel_opt_lm_part_action*)&rhs[1]->user.opt_lm_part_action; +Parser_Lel_longest_match_part *__ref2 = (Parser_Lel_longest_match_part*)&redLel->user.longest_match_part; +Parser_Lel_join *__ref3 = (Parser_Lel_join*)&rhs[0]->user.join; +Token *__ref4 = (Token*)&rhs[2]->user.token; +Parser_Lel_join *__ref5 = (Parser_Lel_join*)&rhs[0]->user.join; +Token *__ref6 = (Token*)&rhs[2]->user.token; +#line 294 "rlparse.kl" + + (__ref0)->lmPart = 0; + Action *action = (__ref1)->action; + if ( action != 0 ) + action->isLmAction = true; + (__ref2)->lmPart = new LongestMatchPart( (__ref3)->join, action, + (__ref4)->loc, pd->nextLongestMatchId++ ); + + /* Provide a location to join. Unfortunately We don't + * have the start of the join as in other occurances. Use the end. */ + (__ref5)->join->loc = (__ref6)->loc; + + +#line 4375 "rlparse.cpp" +} break; +case 43: { +Parser_Lel_opt_lm_part_action *__ref0 = (Parser_Lel_opt_lm_part_action*)&redLel->user.opt_lm_part_action; +Parser_Lel_action_ref *__ref1 = (Parser_Lel_action_ref*)&rhs[1]->user.action_ref; +#line 313 "rlparse.kl" + + (__ref0)->action = (__ref1)->action; + + +#line 4385 "rlparse.cpp" +} break; +case 44: { +Parser_Lel_opt_lm_part_action *__ref0 = (Parser_Lel_opt_lm_part_action*)&redLel->user.opt_lm_part_action; +Parser_Lel_action_ref *__ref1 = (Parser_Lel_action_ref*)&rhs[0]->user.action_ref; +#line 317 "rlparse.kl" + + (__ref0)->action = (__ref1)->action; + + +#line 4395 "rlparse.cpp" +} break; +case 45: { +Parser_Lel_opt_lm_part_action *__ref0 = (Parser_Lel_opt_lm_part_action*)&redLel->user.opt_lm_part_action; +#line 321 "rlparse.kl" + + (__ref0)->action = 0; + + +#line 4404 "rlparse.cpp" +} break; +case 46: { +Parser_Lel_join *__ref0 = (Parser_Lel_join*)&rhs[0]->user.join; +Parser_Lel_expression *__ref1 = (Parser_Lel_expression*)&rhs[2]->user.expression; +Parser_Lel_join *__ref2 = (Parser_Lel_join*)&redLel->user.join; +Parser_Lel_join *__ref3 = (Parser_Lel_join*)&rhs[0]->user.join; +#line 332 "rlparse.kl" + + /* Append the expression to the list and return it. */ + (__ref0)->join->exprList.append( (__ref1)->expression ); + (__ref2)->join = (__ref3)->join; + + +#line 4418 "rlparse.cpp" +} break; +case 47: { +Parser_Lel_join *__ref0 = (Parser_Lel_join*)&redLel->user.join; +Parser_Lel_expression *__ref1 = (Parser_Lel_expression*)&rhs[0]->user.expression; +#line 338 "rlparse.kl" + + (__ref0)->join = new Join( (__ref1)->expression ); + + +#line 4428 "rlparse.cpp" +} break; +case 48: { +Parser_Lel_expression *__ref0 = (Parser_Lel_expression*)&redLel->user.expression; +Parser_Lel_expression *__ref1 = (Parser_Lel_expression*)&rhs[0]->user.expression; +Parser_Lel_term_short *__ref2 = (Parser_Lel_term_short*)&rhs[2]->user.term_short; +#line 348 "rlparse.kl" + + (__ref0)->expression = new Expression( (__ref1)->expression, + (__ref2)->term, Expression::OrType ); + + +#line 4440 "rlparse.cpp" +} break; +case 49: { +Parser_Lel_expression *__ref0 = (Parser_Lel_expression*)&redLel->user.expression; +Parser_Lel_expression *__ref1 = (Parser_Lel_expression*)&rhs[0]->user.expression; +Parser_Lel_term_short *__ref2 = (Parser_Lel_term_short*)&rhs[2]->user.term_short; +#line 353 "rlparse.kl" + + (__ref0)->expression = new Expression( (__ref1)->expression, + (__ref2)->term, Expression::IntersectType ); + + +#line 4452 "rlparse.cpp" +} break; +case 50: { +Parser_Lel_expression *__ref0 = (Parser_Lel_expression*)&redLel->user.expression; +Parser_Lel_expression *__ref1 = (Parser_Lel_expression*)&rhs[0]->user.expression; +Parser_Lel_term_short *__ref2 = (Parser_Lel_term_short*)&rhs[2]->user.term_short; +#line 358 "rlparse.kl" + + (__ref0)->expression = new Expression( (__ref1)->expression, + (__ref2)->term, Expression::SubtractType ); + + +#line 4464 "rlparse.cpp" +} break; +case 51: { +Parser_Lel_expression *__ref0 = (Parser_Lel_expression*)&redLel->user.expression; +Parser_Lel_expression *__ref1 = (Parser_Lel_expression*)&rhs[0]->user.expression; +Parser_Lel_term_short *__ref2 = (Parser_Lel_term_short*)&rhs[2]->user.term_short; +#line 363 "rlparse.kl" + + (__ref0)->expression = new Expression( (__ref1)->expression, + (__ref2)->term, Expression::StrongSubtractType ); + + +#line 4476 "rlparse.cpp" +} break; +case 52: { +Parser_Lel_expression *__ref0 = (Parser_Lel_expression*)&redLel->user.expression; +Parser_Lel_term_short *__ref1 = (Parser_Lel_term_short*)&rhs[0]->user.term_short; +#line 368 "rlparse.kl" + + (__ref0)->expression = new Expression( (__ref1)->term ); + + +#line 4486 "rlparse.cpp" +} break; +case 53: { +Parser_Lel_term_short *__ref0 = (Parser_Lel_term_short*)&redLel->user.term_short; +Parser_Lel_term *__ref1 = (Parser_Lel_term*)&rhs[0]->user.term; +#line 389 "rlparse.kl" + + (__ref0)->term = (__ref1)->term; + + +#line 4496 "rlparse.cpp" +} break; +case 54: { +Parser_Lel_term *__ref0 = (Parser_Lel_term*)&redLel->user.term; +Parser_Lel_term *__ref1 = (Parser_Lel_term*)&rhs[0]->user.term; +Parser_Lel_factor_with_label *__ref2 = (Parser_Lel_factor_with_label*)&rhs[1]->user.factor_with_label; +#line 399 "rlparse.kl" + + (__ref0)->term = new Term( (__ref1)->term, (__ref2)->factorWithAug ); + + +#line 4507 "rlparse.cpp" +} break; +case 55: { +Parser_Lel_term *__ref0 = (Parser_Lel_term*)&redLel->user.term; +Parser_Lel_term *__ref1 = (Parser_Lel_term*)&rhs[0]->user.term; +Parser_Lel_factor_with_label *__ref2 = (Parser_Lel_factor_with_label*)&rhs[2]->user.factor_with_label; +#line 403 "rlparse.kl" + + (__ref0)->term = new Term( (__ref1)->term, (__ref2)->factorWithAug ); + + +#line 4518 "rlparse.cpp" +} break; +case 56: { +Parser_Lel_term *__ref0 = (Parser_Lel_term*)&redLel->user.term; +Parser_Lel_term *__ref1 = (Parser_Lel_term*)&rhs[0]->user.term; +Parser_Lel_factor_with_label *__ref2 = (Parser_Lel_factor_with_label*)&rhs[2]->user.factor_with_label; +#line 407 "rlparse.kl" + + (__ref0)->term = new Term( (__ref1)->term, (__ref2)->factorWithAug, Term::RightStartType ); + + +#line 4529 "rlparse.cpp" +} break; +case 57: { +Parser_Lel_term *__ref0 = (Parser_Lel_term*)&redLel->user.term; +Parser_Lel_term *__ref1 = (Parser_Lel_term*)&rhs[0]->user.term; +Parser_Lel_factor_with_label *__ref2 = (Parser_Lel_factor_with_label*)&rhs[2]->user.factor_with_label; +#line 411 "rlparse.kl" + + (__ref0)->term = new Term( (__ref1)->term, (__ref2)->factorWithAug, Term::RightFinishType ); + + +#line 4540 "rlparse.cpp" +} break; +case 58: { +Parser_Lel_term *__ref0 = (Parser_Lel_term*)&redLel->user.term; +Parser_Lel_term *__ref1 = (Parser_Lel_term*)&rhs[0]->user.term; +Parser_Lel_factor_with_label *__ref2 = (Parser_Lel_factor_with_label*)&rhs[2]->user.factor_with_label; +#line 415 "rlparse.kl" + + (__ref0)->term = new Term( (__ref1)->term, + (__ref2)->factorWithAug, Term::LeftType ); + + +#line 4552 "rlparse.cpp" +} break; +case 59: { +Parser_Lel_term *__ref0 = (Parser_Lel_term*)&redLel->user.term; +Parser_Lel_factor_with_label *__ref1 = (Parser_Lel_factor_with_label*)&rhs[0]->user.factor_with_label; +#line 420 "rlparse.kl" + + (__ref0)->term = new Term( (__ref1)->factorWithAug ); + + +#line 4562 "rlparse.cpp" +} break; +case 60: { +Parser_Lel_factor_with_label *__ref0 = (Parser_Lel_factor_with_label*)&rhs[2]->user.factor_with_label; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Token *__ref2 = (Token*)&rhs[0]->user.token; +Parser_Lel_factor_with_label *__ref3 = (Parser_Lel_factor_with_label*)&redLel->user.factor_with_label; +Parser_Lel_factor_with_label *__ref4 = (Parser_Lel_factor_with_label*)&rhs[2]->user.factor_with_label; +#line 430 "rlparse.kl" + + /* Add the label to the list and pass the factor up. */ + (__ref0)->factorWithAug->labels.prepend( Label((__ref1)->loc, (__ref2)->data) ); + (__ref3)->factorWithAug = (__ref4)->factorWithAug; + + +#line 4577 "rlparse.cpp" +} break; +case 61: { +Parser_Lel_factor_with_label *__ref0 = (Parser_Lel_factor_with_label*)&redLel->user.factor_with_label; +Parser_Lel_factor_with_ep *__ref1 = (Parser_Lel_factor_with_ep*)&rhs[0]->user.factor_with_ep; +#line 436 "rlparse.kl" + + (__ref0)->factorWithAug = (__ref1)->factorWithAug; + + +#line 4587 "rlparse.cpp" +} break; +case 62: { +Parser_Lel_factor_with_ep *__ref0 = (Parser_Lel_factor_with_ep*)&rhs[0]->user.factor_with_ep; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Parser_Lel_factor_with_ep *__ref2 = (Parser_Lel_factor_with_ep*)&redLel->user.factor_with_ep; +Parser_Lel_factor_with_ep *__ref3 = (Parser_Lel_factor_with_ep*)&rhs[0]->user.factor_with_ep; +#line 446 "rlparse.kl" + + /* Add the target to the list and return the factor object. */ + (__ref0)->factorWithAug->epsilonLinks.append( EpsilonLink( (__ref1)->loc, nameRef ) ); + (__ref2)->factorWithAug = (__ref3)->factorWithAug; + + +#line 4601 "rlparse.cpp" +} break; +case 63: { +Parser_Lel_factor_with_ep *__ref0 = (Parser_Lel_factor_with_ep*)&redLel->user.factor_with_ep; +Parser_Lel_factor_with_aug *__ref1 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 452 "rlparse.kl" + + (__ref0)->factorWithAug = (__ref1)->factorWithAug; + + +#line 4611 "rlparse.cpp" +} break; +case 64: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_action_ref *__ref3 = (Parser_Lel_action_ref*)&rhs[2]->user.action_ref; +Parser_Lel_factor_with_aug *__ref4 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref5 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 462 "rlparse.kl" + + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + (__ref0)->factorWithAug->actions.append( + ParserAction( (__ref1)->loc, (__ref2)->augType, 0, (__ref3)->action ) ); + (__ref4)->factorWithAug = (__ref5)->factorWithAug; + + +#line 4629 "rlparse.cpp" +} break; +case 65: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_priority_aug *__ref2 = (Parser_Lel_priority_aug*)&rhs[2]->user.priority_aug; +Parser_Lel_factor_with_aug *__ref3 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref4 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 470 "rlparse.kl" + + /* Append the named priority to the factorWithAug and pass it up. */ + (__ref0)->factorWithAug->priorityAugs.append( + PriorityAug( (__ref1)->augType, pd->curDefPriorKey, (__ref2)->priorityNum ) ); + (__ref3)->factorWithAug = (__ref4)->factorWithAug; + + +#line 4645 "rlparse.cpp" +} break; +case 66: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_priority_name *__ref2 = (Parser_Lel_priority_name*)&rhs[3]->user.priority_name; +Parser_Lel_priority_aug *__ref3 = (Parser_Lel_priority_aug*)&rhs[5]->user.priority_aug; +Parser_Lel_factor_with_aug *__ref4 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref5 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 477 "rlparse.kl" + + /* Append the priority using a default name. */ + (__ref0)->factorWithAug->priorityAugs.append( + PriorityAug( (__ref1)->augType, (__ref2)->priorityName, (__ref3)->priorityNum ) ); + (__ref4)->factorWithAug = (__ref5)->factorWithAug; + + +#line 4662 "rlparse.cpp" +} break; +case 67: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_action_ref *__ref3 = (Parser_Lel_action_ref*)&rhs[2]->user.action_ref; +Parser_Lel_factor_with_aug *__ref4 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref5 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 484 "rlparse.kl" + + (__ref0)->factorWithAug->conditions.append( ConditionTest( (__ref1)->loc, + (__ref2)->augType, (__ref3)->action, true ) ); + (__ref4)->factorWithAug = (__ref5)->factorWithAug; + + +#line 4678 "rlparse.cpp" +} break; +case 68: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_action_ref *__ref3 = (Parser_Lel_action_ref*)&rhs[3]->user.action_ref; +Parser_Lel_factor_with_aug *__ref4 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref5 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 490 "rlparse.kl" + + (__ref0)->factorWithAug->conditions.append( ConditionTest( (__ref1)->loc, + (__ref2)->augType, (__ref3)->action, false ) ); + (__ref4)->factorWithAug = (__ref5)->factorWithAug; + + +#line 4694 "rlparse.cpp" +} break; +case 69: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_action_ref *__ref3 = (Parser_Lel_action_ref*)&rhs[2]->user.action_ref; +Parser_Lel_factor_with_aug *__ref4 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref5 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 496 "rlparse.kl" + + /* Append the action, pass it up. */ + (__ref0)->factorWithAug->actions.append( ParserAction( (__ref1)->loc, + (__ref2)->augType, 0, (__ref3)->action ) ); + (__ref4)->factorWithAug = (__ref5)->factorWithAug; + + +#line 4711 "rlparse.cpp" +} break; +case 70: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_action_ref *__ref3 = (Parser_Lel_action_ref*)&rhs[2]->user.action_ref; +Parser_Lel_factor_with_aug *__ref4 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref5 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 503 "rlparse.kl" + + /* Append the action, pass it up. */ + (__ref0)->factorWithAug->actions.append( ParserAction( (__ref1)->loc, + (__ref2)->augType, 0, (__ref3)->action ) ); + (__ref4)->factorWithAug = (__ref5)->factorWithAug; + + +#line 4728 "rlparse.cpp" +} break; +case 71: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_action_ref *__ref3 = (Parser_Lel_action_ref*)&rhs[2]->user.action_ref; +Parser_Lel_factor_with_aug *__ref4 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref5 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 510 "rlparse.kl" + + /* Append the action, pass it up. */ + (__ref0)->factorWithAug->actions.append( ParserAction( (__ref1)->loc, + (__ref2)->augType, 0, (__ref3)->action ) ); + (__ref4)->factorWithAug = (__ref5)->factorWithAug; + + +#line 4745 "rlparse.cpp" +} break; +case 72: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_action_ref *__ref3 = (Parser_Lel_action_ref*)&rhs[2]->user.action_ref; +Parser_Lel_factor_with_aug *__ref4 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref5 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 517 "rlparse.kl" + + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + (__ref0)->factorWithAug->actions.append( ParserAction( (__ref1)->loc, + (__ref2)->augType, pd->curDefLocalErrKey, (__ref3)->action ) ); + (__ref4)->factorWithAug = (__ref5)->factorWithAug; + + +#line 4763 "rlparse.cpp" +} break; +case 73: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_action_ref *__ref3 = (Parser_Lel_action_ref*)&rhs[2]->user.action_ref; +Parser_Lel_factor_with_aug *__ref4 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref5 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 525 "rlparse.kl" + + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + (__ref0)->factorWithAug->actions.append( ParserAction( (__ref1)->loc, + (__ref2)->augType, pd->curDefLocalErrKey, (__ref3)->action ) ); + (__ref4)->factorWithAug = (__ref5)->factorWithAug; + + +#line 4781 "rlparse.cpp" +} break; +case 74: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +Parser_Lel_aug_type *__ref1 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&rhs[1]->user.aug_type; +Parser_Lel_local_err_name *__ref3 = (Parser_Lel_local_err_name*)&rhs[3]->user.local_err_name; +Parser_Lel_action_ref *__ref4 = (Parser_Lel_action_ref*)&rhs[5]->user.action_ref; +Parser_Lel_factor_with_aug *__ref5 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_aug *__ref6 = (Parser_Lel_factor_with_aug*)&rhs[0]->user.factor_with_aug; +#line 533 "rlparse.kl" + + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + (__ref0)->factorWithAug->actions.append( ParserAction( (__ref1)->loc, + (__ref2)->augType, (__ref3)->error_name, (__ref4)->action ) ); + (__ref5)->factorWithAug = (__ref6)->factorWithAug; + + +#line 4800 "rlparse.cpp" +} break; +case 75: { +Parser_Lel_factor_with_aug *__ref0 = (Parser_Lel_factor_with_aug*)&redLel->user.factor_with_aug; +Parser_Lel_factor_with_rep *__ref1 = (Parser_Lel_factor_with_rep*)&rhs[0]->user.factor_with_rep; +#line 541 "rlparse.kl" + + (__ref0)->factorWithAug = new FactorWithAug( (__ref1)->factorWithRep ); + + +#line 4810 "rlparse.cpp" +} break; +case 76: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 554 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_finish; + +#line 4819 "rlparse.cpp" +} break; +case 77: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 555 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_leave; + +#line 4828 "rlparse.cpp" +} break; +case 78: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 556 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all; + +#line 4837 "rlparse.cpp" +} break; +case 79: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 557 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start; + +#line 4846 "rlparse.cpp" +} break; +case 80: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 562 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start; + +#line 4855 "rlparse.cpp" +} break; +case 81: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 563 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start; + +#line 4864 "rlparse.cpp" +} break; +case 82: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 564 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all; + +#line 4873 "rlparse.cpp" +} break; +case 83: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 565 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all; + +#line 4882 "rlparse.cpp" +} break; +case 84: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 566 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_leave; + +#line 4891 "rlparse.cpp" +} break; +case 85: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 567 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_leave; + +#line 4900 "rlparse.cpp" +} break; +case 86: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 568 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all; + +#line 4909 "rlparse.cpp" +} break; +case 87: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 569 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start; + +#line 4918 "rlparse.cpp" +} break; +case 88: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 570 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_leave; + +#line 4927 "rlparse.cpp" +} break; +case 89: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 579 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start_to_state; + +#line 4936 "rlparse.cpp" +} break; +case 90: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 581 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start_to_state; + +#line 4945 "rlparse.cpp" +} break; +case 91: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 584 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_start_to_state; + +#line 4954 "rlparse.cpp" +} break; +case 92: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 586 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_start_to_state; + +#line 4963 "rlparse.cpp" +} break; +case 93: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 589 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all_to_state; + +#line 4972 "rlparse.cpp" +} break; +case 94: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 591 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all_to_state; + +#line 4981 "rlparse.cpp" +} break; +case 95: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 594 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_final_to_state; + +#line 4990 "rlparse.cpp" +} break; +case 96: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 596 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_final_to_state; + +#line 4999 "rlparse.cpp" +} break; +case 97: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 599 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_final_to_state; + +#line 5008 "rlparse.cpp" +} break; +case 98: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 601 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_final_to_state; + +#line 5017 "rlparse.cpp" +} break; +case 99: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 604 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_middle_to_state; + +#line 5026 "rlparse.cpp" +} break; +case 100: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 606 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_middle_to_state; + +#line 5035 "rlparse.cpp" +} break; +case 101: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 615 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start_from_state; + +#line 5044 "rlparse.cpp" +} break; +case 102: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 617 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start_from_state; + +#line 5053 "rlparse.cpp" +} break; +case 103: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 620 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_start_from_state; + +#line 5062 "rlparse.cpp" +} break; +case 104: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 622 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_start_from_state; + +#line 5071 "rlparse.cpp" +} break; +case 105: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 625 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all_from_state; + +#line 5080 "rlparse.cpp" +} break; +case 106: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 627 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all_from_state; + +#line 5089 "rlparse.cpp" +} break; +case 107: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 630 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_final_from_state; + +#line 5098 "rlparse.cpp" +} break; +case 108: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 632 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_final_from_state; + +#line 5107 "rlparse.cpp" +} break; +case 109: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 635 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_final_from_state; + +#line 5116 "rlparse.cpp" +} break; +case 110: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 637 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_final_from_state; + +#line 5125 "rlparse.cpp" +} break; +case 111: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 640 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_middle_from_state; + +#line 5134 "rlparse.cpp" +} break; +case 112: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 642 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_middle_from_state; + +#line 5143 "rlparse.cpp" +} break; +case 113: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 651 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start_eof; + +#line 5152 "rlparse.cpp" +} break; +case 114: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 653 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start_eof; + +#line 5161 "rlparse.cpp" +} break; +case 115: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 656 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_start_eof; + +#line 5170 "rlparse.cpp" +} break; +case 116: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 658 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_start_eof; + +#line 5179 "rlparse.cpp" +} break; +case 117: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 661 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all_eof; + +#line 5188 "rlparse.cpp" +} break; +case 118: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 663 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all_eof; + +#line 5197 "rlparse.cpp" +} break; +case 119: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 666 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_final_eof; + +#line 5206 "rlparse.cpp" +} break; +case 120: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 668 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_final_eof; + +#line 5215 "rlparse.cpp" +} break; +case 121: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 671 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_final_eof; + +#line 5224 "rlparse.cpp" +} break; +case 122: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 673 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_final_eof; + +#line 5233 "rlparse.cpp" +} break; +case 123: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 676 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_middle_eof; + +#line 5242 "rlparse.cpp" +} break; +case 124: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 678 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_middle_eof; + +#line 5251 "rlparse.cpp" +} break; +case 125: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 687 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start_gbl_error; + +#line 5260 "rlparse.cpp" +} break; +case 126: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 689 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start_gbl_error; + +#line 5269 "rlparse.cpp" +} break; +case 127: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 692 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_start_gbl_error; + +#line 5278 "rlparse.cpp" +} break; +case 128: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 694 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_start_gbl_error; + +#line 5287 "rlparse.cpp" +} break; +case 129: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 697 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all_gbl_error; + +#line 5296 "rlparse.cpp" +} break; +case 130: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 699 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all_gbl_error; + +#line 5305 "rlparse.cpp" +} break; +case 131: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 702 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_final_gbl_error; + +#line 5314 "rlparse.cpp" +} break; +case 132: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 704 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_final_gbl_error; + +#line 5323 "rlparse.cpp" +} break; +case 133: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 707 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_final_gbl_error; + +#line 5332 "rlparse.cpp" +} break; +case 134: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 709 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_final_gbl_error; + +#line 5341 "rlparse.cpp" +} break; +case 135: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 712 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_middle_gbl_error; + +#line 5350 "rlparse.cpp" +} break; +case 136: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 714 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_middle_gbl_error; + +#line 5359 "rlparse.cpp" +} break; +case 137: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 724 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start_local_error; + +#line 5368 "rlparse.cpp" +} break; +case 138: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 726 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_start_local_error; + +#line 5377 "rlparse.cpp" +} break; +case 139: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 729 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_start_local_error; + +#line 5386 "rlparse.cpp" +} break; +case 140: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 731 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_start_local_error; + +#line 5395 "rlparse.cpp" +} break; +case 141: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 734 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all_local_error; + +#line 5404 "rlparse.cpp" +} break; +case 142: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 736 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_all_local_error; + +#line 5413 "rlparse.cpp" +} break; +case 143: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 739 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_final_local_error; + +#line 5422 "rlparse.cpp" +} break; +case 144: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 741 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_final_local_error; + +#line 5431 "rlparse.cpp" +} break; +case 145: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 744 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_final_local_error; + +#line 5440 "rlparse.cpp" +} break; +case 146: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 746 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_not_final_local_error; + +#line 5449 "rlparse.cpp" +} break; +case 147: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 749 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_middle_local_error; + +#line 5458 "rlparse.cpp" +} break; +case 148: { +Parser_Lel_aug_type *__ref0 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_aug_type *__ref2 = (Parser_Lel_aug_type*)&redLel->user.aug_type; +#line 751 "rlparse.kl" + (__ref0)->loc = (__ref1)->loc; (__ref2)->augType = at_middle_local_error; + +#line 5467 "rlparse.cpp" +} break; +case 149: { +Parser_Lel_action_ref *__ref0 = (Parser_Lel_action_ref*)&redLel->user.action_ref; +Parser_Lel_action_ref *__ref1 = (Parser_Lel_action_ref*)&rhs[0]->user.action_ref; +#line 764 "rlparse.kl" + (__ref0)->action = (__ref1)->action; + +#line 5475 "rlparse.cpp" +} break; +case 150: { +Parser_Lel_action_ref *__ref0 = (Parser_Lel_action_ref*)&redLel->user.action_ref; +Parser_Lel_action_ref *__ref1 = (Parser_Lel_action_ref*)&rhs[1]->user.action_ref; +#line 765 "rlparse.kl" + (__ref0)->action = (__ref1)->action; + +#line 5483 "rlparse.cpp" +} break; +case 151: { +Parser_Lel_action_ref *__ref0 = (Parser_Lel_action_ref*)&redLel->user.action_ref; +Parser_Lel_action_ref *__ref1 = (Parser_Lel_action_ref*)&rhs[0]->user.action_ref; +#line 766 "rlparse.kl" + (__ref0)->action = (__ref1)->action; + +#line 5491 "rlparse.cpp" +} break; +case 152: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +Parser_Lel_action_ref *__ref1 = (Parser_Lel_action_ref*)&redLel->user.action_ref; +Token *__ref2 = (Token*)&rhs[0]->user.token; +Token *__ref3 = (Token*)&rhs[0]->user.token; +Parser_Lel_action_ref *__ref4 = (Parser_Lel_action_ref*)&redLel->user.action_ref; +#line 771 "rlparse.kl" + + /* Set the name in the actionDict. */ + Action *action = pd->actionDict.find( (__ref0)->data ); + if ( action != 0 ) { + /* Pass up the action element */ + (__ref1)->action = action; + } + else { + /* Will recover by returning null as the action. */ + error((__ref2)->loc) << "action lookup of \"" << (__ref3)->data << "\" failed" << endl; + (__ref4)->action = 0; + } + + +#line 5514 "rlparse.cpp" +} break; +case 153: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +Parser_Lel_inline_list *__ref1 = (Parser_Lel_inline_list*)&rhs[1]->user.inline_list; +Parser_Lel_action_ref *__ref2 = (Parser_Lel_action_ref*)&redLel->user.action_ref; +#line 788 "rlparse.kl" + + /* Create the action, add it to the list and pass up. */ + Action *newAction = new Action( (__ref0)->loc, 0, (__ref1)->inlineList, pd->nextCondId++ ); + pd->actionList.append( newAction ); + (__ref2)->action = newAction; + + +#line 5528 "rlparse.cpp" +} break; +case 154: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +Parser_Lel_priority_name *__ref1 = (Parser_Lel_priority_name*)&redLel->user.priority_name; +#line 803 "rlparse.kl" + + // Lookup/create the priority key. + PriorDictEl *priorDictEl; + if ( pd->priorDict.insert( (__ref0)->data, pd->nextPriorKey, &priorDictEl ) ) + pd->nextPriorKey += 1; + + // Use the inserted/found priority key. + (__ref1)->priorityName = priorDictEl->value; + + +#line 5544 "rlparse.cpp" +} break; +case 155: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +Parser_Lel_token_type *__ref2 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +Parser_Lel_priority_aug *__ref3 = (Parser_Lel_priority_aug*)&redLel->user.priority_aug; +Parser_Lel_token_type *__ref4 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +Parser_Lel_token_type *__ref5 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +Parser_Lel_priority_aug *__ref6 = (Parser_Lel_priority_aug*)&redLel->user.priority_aug; +Parser_Lel_priority_aug *__ref7 = (Parser_Lel_priority_aug*)&redLel->user.priority_aug; +#line 820 "rlparse.kl" + + // Convert the priority number to a long. Check for overflow. + errno = 0; + //cerr << "PRIOR AUG: " << $1->token.data << endl; + long aug = strtol( (__ref0)->token.data, 0, 10 ); + if ( errno == ERANGE && aug == LONG_MAX ) { + /* Priority number too large. Recover by setting the priority to 0. */ + error((__ref1)->token.loc) << "priority number " << (__ref2)->token.data << + " overflows" << endl; + (__ref3)->priorityNum = 0; + } + else if ( errno == ERANGE && aug == LONG_MIN ) { + /* Priority number too large in the neg. Recover by using 0. */ + error((__ref4)->token.loc) << "priority number " << (__ref5)->token.data << + " underflows" << endl; + (__ref6)->priorityNum = 0; + } + else { + /* No overflow or underflow. */ + (__ref7)->priorityNum = aug; + } + + +#line 5579 "rlparse.cpp" +} break; +case 156: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 846 "rlparse.kl" + + (__ref0)->token = *(__ref1); + + +#line 5589 "rlparse.cpp" +} break; +case 157: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref2 = (Token*)&rhs[0]->user.token; +Parser_Lel_token_type *__ref3 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref4 = (Token*)&rhs[1]->user.token; +#line 850 "rlparse.kl" + + (__ref0)->token.set( "+", 1 ); + (__ref1)->token.loc = (__ref2)->loc; + (__ref3)->token.append( *(__ref4) ); + + +#line 5604 "rlparse.cpp" +} break; +case 158: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref2 = (Token*)&rhs[0]->user.token; +Parser_Lel_token_type *__ref3 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref4 = (Token*)&rhs[1]->user.token; +#line 856 "rlparse.kl" + + (__ref0)->token.set( "-", 1 ); + (__ref1)->token.loc = (__ref2)->loc; + (__ref3)->token.append( *(__ref4) ); + + +#line 5619 "rlparse.cpp" +} break; +case 159: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +Parser_Lel_local_err_name *__ref1 = (Parser_Lel_local_err_name*)&redLel->user.local_err_name; +#line 868 "rlparse.kl" + + /* Lookup/create the priority key. */ + LocalErrDictEl *localErrDictEl; + if ( pd->localErrDict.insert( (__ref0)->data, pd->nextLocalErrKey, &localErrDictEl ) ) + pd->nextLocalErrKey += 1; + + /* Use the inserted/found priority key. */ + (__ref1)->error_name = localErrDictEl->value; + + +#line 5635 "rlparse.cpp" +} break; +case 160: { +Parser_Lel_factor_with_rep *__ref0 = (Parser_Lel_factor_with_rep*)&redLel->user.factor_with_rep; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Parser_Lel_factor_with_rep *__ref2 = (Parser_Lel_factor_with_rep*)&rhs[0]->user.factor_with_rep; +#line 889 "rlparse.kl" + + (__ref0)->factorWithRep = new FactorWithRep( (__ref1)->loc, (__ref2)->factorWithRep, + 0, 0, FactorWithRep::StarType ); + + +#line 5647 "rlparse.cpp" +} break; +case 161: { +Parser_Lel_factor_with_rep *__ref0 = (Parser_Lel_factor_with_rep*)&redLel->user.factor_with_rep; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Parser_Lel_factor_with_rep *__ref2 = (Parser_Lel_factor_with_rep*)&rhs[0]->user.factor_with_rep; +#line 894 "rlparse.kl" + + (__ref0)->factorWithRep = new FactorWithRep( (__ref1)->loc, (__ref2)->factorWithRep, + 0, 0, FactorWithRep::StarStarType ); + + +#line 5659 "rlparse.cpp" +} break; +case 162: { +Parser_Lel_factor_with_rep *__ref0 = (Parser_Lel_factor_with_rep*)&redLel->user.factor_with_rep; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Parser_Lel_factor_with_rep *__ref2 = (Parser_Lel_factor_with_rep*)&rhs[0]->user.factor_with_rep; +#line 899 "rlparse.kl" + + (__ref0)->factorWithRep = new FactorWithRep( (__ref1)->loc, (__ref2)->factorWithRep, + 0, 0, FactorWithRep::OptionalType ); + + +#line 5671 "rlparse.cpp" +} break; +case 163: { +Parser_Lel_factor_with_rep *__ref0 = (Parser_Lel_factor_with_rep*)&redLel->user.factor_with_rep; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Parser_Lel_factor_with_rep *__ref2 = (Parser_Lel_factor_with_rep*)&rhs[0]->user.factor_with_rep; +#line 904 "rlparse.kl" + + (__ref0)->factorWithRep = new FactorWithRep( (__ref1)->loc, (__ref2)->factorWithRep, + 0, 0, FactorWithRep::PlusType ); + + +#line 5683 "rlparse.cpp" +} break; +case 164: { +Parser_Lel_factor_with_rep *__ref0 = (Parser_Lel_factor_with_rep*)&redLel->user.factor_with_rep; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Parser_Lel_factor_with_rep *__ref2 = (Parser_Lel_factor_with_rep*)&rhs[0]->user.factor_with_rep; +Parser_Lel_factor_rep_num *__ref3 = (Parser_Lel_factor_rep_num*)&rhs[2]->user.factor_rep_num; +#line 909 "rlparse.kl" + + (__ref0)->factorWithRep = new FactorWithRep( (__ref1)->loc, (__ref2)->factorWithRep, + (__ref3)->rep, 0, FactorWithRep::ExactType ); + + +#line 5696 "rlparse.cpp" +} break; +case 165: { +Parser_Lel_factor_with_rep *__ref0 = (Parser_Lel_factor_with_rep*)&redLel->user.factor_with_rep; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Parser_Lel_factor_with_rep *__ref2 = (Parser_Lel_factor_with_rep*)&rhs[0]->user.factor_with_rep; +Parser_Lel_factor_rep_num *__ref3 = (Parser_Lel_factor_rep_num*)&rhs[3]->user.factor_rep_num; +#line 914 "rlparse.kl" + + (__ref0)->factorWithRep = new FactorWithRep( (__ref1)->loc, (__ref2)->factorWithRep, + 0, (__ref3)->rep, FactorWithRep::MaxType ); + + +#line 5709 "rlparse.cpp" +} break; +case 166: { +Parser_Lel_factor_with_rep *__ref0 = (Parser_Lel_factor_with_rep*)&redLel->user.factor_with_rep; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Parser_Lel_factor_with_rep *__ref2 = (Parser_Lel_factor_with_rep*)&rhs[0]->user.factor_with_rep; +Parser_Lel_factor_rep_num *__ref3 = (Parser_Lel_factor_rep_num*)&rhs[2]->user.factor_rep_num; +#line 919 "rlparse.kl" + + (__ref0)->factorWithRep = new FactorWithRep( (__ref1)->loc, (__ref2)->factorWithRep, + (__ref3)->rep, 0, FactorWithRep::MinType ); + + +#line 5722 "rlparse.cpp" +} break; +case 167: { +Parser_Lel_factor_with_rep *__ref0 = (Parser_Lel_factor_with_rep*)&redLel->user.factor_with_rep; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Parser_Lel_factor_with_rep *__ref2 = (Parser_Lel_factor_with_rep*)&rhs[0]->user.factor_with_rep; +Parser_Lel_factor_rep_num *__ref3 = (Parser_Lel_factor_rep_num*)&rhs[2]->user.factor_rep_num; +Parser_Lel_factor_rep_num *__ref4 = (Parser_Lel_factor_rep_num*)&rhs[4]->user.factor_rep_num; +#line 924 "rlparse.kl" + + (__ref0)->factorWithRep = new FactorWithRep( (__ref1)->loc, (__ref2)->factorWithRep, + (__ref3)->rep, (__ref4)->rep, FactorWithRep::RangeType ); + + +#line 5736 "rlparse.cpp" +} break; +case 168: { +Parser_Lel_factor_with_rep *__ref0 = (Parser_Lel_factor_with_rep*)&redLel->user.factor_with_rep; +Parser_Lel_factor_with_neg *__ref1 = (Parser_Lel_factor_with_neg*)&rhs[0]->user.factor_with_neg; +#line 929 "rlparse.kl" + + (__ref0)->factorWithRep = new FactorWithRep( (__ref1)->factorWithNeg ); + + +#line 5746 "rlparse.cpp" +} break; +case 169: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Token *__ref2 = (Token*)&rhs[0]->user.token; +Parser_Lel_factor_rep_num *__ref3 = (Parser_Lel_factor_rep_num*)&redLel->user.factor_rep_num; +Parser_Lel_factor_rep_num *__ref4 = (Parser_Lel_factor_rep_num*)&redLel->user.factor_rep_num; +#line 939 "rlparse.kl" + + // Convert the priority number to a long. Check for overflow. + errno = 0; + long rep = strtol( (__ref0)->data, 0, 10 ); + if ( errno == ERANGE && rep == LONG_MAX ) { + // Repetition too large. Recover by returing repetition 1. */ + error((__ref1)->loc) << "repetition number " << (__ref2)->data << " overflows" << endl; + (__ref3)->rep = 1; + } + else { + // Cannot be negative, so no overflow. + (__ref4)->rep = rep; + } + + +#line 5770 "rlparse.cpp" +} break; +case 170: { +Parser_Lel_factor_with_neg *__ref0 = (Parser_Lel_factor_with_neg*)&redLel->user.factor_with_neg; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_factor_with_neg *__ref2 = (Parser_Lel_factor_with_neg*)&rhs[1]->user.factor_with_neg; +#line 965 "rlparse.kl" + + (__ref0)->factorWithNeg = new FactorWithNeg( (__ref1)->loc, + (__ref2)->factorWithNeg, FactorWithNeg::NegateType ); + + +#line 5782 "rlparse.cpp" +} break; +case 171: { +Parser_Lel_factor_with_neg *__ref0 = (Parser_Lel_factor_with_neg*)&redLel->user.factor_with_neg; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_factor_with_neg *__ref2 = (Parser_Lel_factor_with_neg*)&rhs[1]->user.factor_with_neg; +#line 970 "rlparse.kl" + + (__ref0)->factorWithNeg = new FactorWithNeg( (__ref1)->loc, + (__ref2)->factorWithNeg, FactorWithNeg::CharNegateType ); + + +#line 5794 "rlparse.cpp" +} break; +case 172: { +Parser_Lel_factor_with_neg *__ref0 = (Parser_Lel_factor_with_neg*)&redLel->user.factor_with_neg; +Parser_Lel_factor *__ref1 = (Parser_Lel_factor*)&rhs[0]->user.factor; +#line 975 "rlparse.kl" + + (__ref0)->factorWithNeg = new FactorWithNeg( (__ref1)->factor ); + + +#line 5804 "rlparse.cpp" +} break; +case 173: { +Parser_Lel_factor *__ref0 = (Parser_Lel_factor*)&redLel->user.factor; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 985 "rlparse.kl" + + /* Create a new factor node going to a concat literal. */ + (__ref0)->factor = new Factor( new Literal( *(__ref1), Literal::LitString ) ); + + +#line 5815 "rlparse.cpp" +} break; +case 174: { +Parser_Lel_factor *__ref0 = (Parser_Lel_factor*)&redLel->user.factor; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +#line 990 "rlparse.kl" + + /* Create a new factor node going to a literal number. */ + (__ref0)->factor = new Factor( new Literal( (__ref1)->token, Literal::Number ) ); + + +#line 5826 "rlparse.cpp" +} break; +case 175: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Token *__ref2 = (Token*)&rhs[0]->user.token; +Parser_Lel_factor *__ref3 = (Parser_Lel_factor*)&redLel->user.factor; +Token *__ref4 = (Token*)&rhs[0]->user.token; +Parser_Lel_factor *__ref5 = (Parser_Lel_factor*)&redLel->user.factor; +Parser_Lel_factor *__ref6 = (Parser_Lel_factor*)&redLel->user.factor; +Token *__ref7 = (Token*)&rhs[0]->user.token; +#line 995 "rlparse.kl" + + /* Find the named graph. */ + GraphDictEl *gdNode = pd->graphDict.find( (__ref0)->data ); + if ( gdNode == 0 ) { + /* Recover by returning null as the factor node. */ + error((__ref1)->loc) << "graph lookup of \"" << (__ref2)->data << "\" failed" << endl; + (__ref3)->factor = 0; + } + else if ( gdNode->isInstance ) { + /* Recover by retuning null as the factor node. */ + error((__ref4)->loc) << "references to graph instantiations not allowed " + "in expressions" << endl; + (__ref5)->factor = 0; + } + else { + /* Create a factor node that is a lookup of an expression. */ + (__ref6)->factor = new Factor( (__ref7)->loc, gdNode->value ); + } + + +#line 5858 "rlparse.cpp" +} break; +case 176: { +Parser_Lel_factor *__ref0 = (Parser_Lel_factor*)&redLel->user.factor; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_regular_expr_or_data *__ref2 = (Parser_Lel_regular_expr_or_data*)&rhs[1]->user.regular_expr_or_data; +#line 1015 "rlparse.kl" + + /* Create a new factor node going to an OR expression. */ + (__ref0)->factor = new Factor( new ReItem( (__ref1)->loc, (__ref2)->reOrBlock, ReItem::OrBlock ) ); + + +#line 5870 "rlparse.cpp" +} break; +case 177: { +Parser_Lel_factor *__ref0 = (Parser_Lel_factor*)&redLel->user.factor; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_regular_expr_or_data *__ref2 = (Parser_Lel_regular_expr_or_data*)&rhs[1]->user.regular_expr_or_data; +#line 1020 "rlparse.kl" + + /* Create a new factor node going to a negated OR expression. */ + (__ref0)->factor = new Factor( new ReItem( (__ref1)->loc, (__ref2)->reOrBlock, ReItem::NegOrBlock ) ); + + +#line 5882 "rlparse.cpp" +} break; +case 178: { +Token *__ref0 = (Token*)&rhs[2]->user.token; +Token *__ref1 = (Token*)&rhs[2]->user.token; +Parser_Lel_regular_expr *__ref2 = (Parser_Lel_regular_expr*)&rhs[1]->user.regular_expr; +Parser_Lel_factor *__ref3 = (Parser_Lel_factor*)&redLel->user.factor; +Parser_Lel_regular_expr *__ref4 = (Parser_Lel_regular_expr*)&rhs[1]->user.regular_expr; +#line 1025 "rlparse.kl" + + if ( (__ref0)->length > 1 ) { + for ( char *p = (__ref1)->data; *p != 0; p++ ) { + if ( *p == 'i' ) + (__ref2)->regExpr->caseInsensitive = true; + } + } + + /* Create a new factor node going to a regular exp. */ + (__ref3)->factor = new Factor( (__ref4)->regExpr ); + + +#line 5903 "rlparse.cpp" +} break; +case 179: { +Parser_Lel_factor *__ref0 = (Parser_Lel_factor*)&redLel->user.factor; +Parser_Lel_range_lit *__ref1 = (Parser_Lel_range_lit*)&rhs[0]->user.range_lit; +Parser_Lel_range_lit *__ref2 = (Parser_Lel_range_lit*)&rhs[2]->user.range_lit; +#line 1037 "rlparse.kl" + + /* Create a new factor node going to a range. */ + (__ref0)->factor = new Factor( new Range( (__ref1)->literal, (__ref2)->literal ) ); + + +#line 5915 "rlparse.cpp" +} break; +case 180: { +Parser_Lel_factor *__ref0 = (Parser_Lel_factor*)&redLel->user.factor; +Parser_Lel_join *__ref1 = (Parser_Lel_join*)&rhs[1]->user.join; +Parser_Lel_join *__ref2 = (Parser_Lel_join*)&rhs[1]->user.join; +Token *__ref3 = (Token*)&rhs[0]->user.token; +#line 1042 "rlparse.kl" + + /* Create a new factor going to a parenthesized join. */ + (__ref0)->factor = new Factor( (__ref1)->join ); + (__ref2)->join->loc = (__ref3)->loc; + + +#line 5929 "rlparse.cpp" +} break; +case 181: { +Parser_Lel_range_lit *__ref0 = (Parser_Lel_range_lit*)&redLel->user.range_lit; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1055 "rlparse.kl" + + /* Range literas must have only one char. We restrict this in the parse tree. */ + (__ref0)->literal = new Literal( *(__ref1), Literal::LitString ); + + +#line 5940 "rlparse.cpp" +} break; +case 182: { +Parser_Lel_range_lit *__ref0 = (Parser_Lel_range_lit*)&redLel->user.range_lit; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +#line 1060 "rlparse.kl" + + /* Create a new literal number. */ + (__ref0)->literal = new Literal( (__ref1)->token, Literal::Number ); + + +#line 5951 "rlparse.cpp" +} break; +case 183: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1069 "rlparse.kl" + + (__ref0)->token = *(__ref1); + + +#line 5961 "rlparse.cpp" +} break; +case 184: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref2 = (Token*)&rhs[0]->user.token; +Parser_Lel_token_type *__ref3 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref4 = (Token*)&rhs[1]->user.token; +#line 1073 "rlparse.kl" + + (__ref0)->token.set( "-", 1 ); + (__ref1)->token.loc = (__ref2)->loc; + (__ref3)->token.append( *(__ref4) ); + + +#line 5976 "rlparse.cpp" +} break; +case 185: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1079 "rlparse.kl" + + (__ref0)->token = *(__ref1); + + +#line 5986 "rlparse.cpp" +} break; +case 186: { +Parser_Lel_regular_expr_item *__ref0 = (Parser_Lel_regular_expr_item*)&rhs[1]->user.regular_expr_item; +Parser_Lel_regular_expr_item *__ref1 = (Parser_Lel_regular_expr_item*)&rhs[1]->user.regular_expr_item; +Parser_Lel_regular_expr *__ref2 = (Parser_Lel_regular_expr*)&rhs[0]->user.regular_expr; +Parser_Lel_regular_expr *__ref3 = (Parser_Lel_regular_expr*)&rhs[0]->user.regular_expr; +Parser_Lel_regular_expr *__ref4 = (Parser_Lel_regular_expr*)&rhs[0]->user.regular_expr; +Parser_Lel_regular_expr *__ref5 = (Parser_Lel_regular_expr*)&rhs[0]->user.regular_expr; +Parser_Lel_regular_expr_item *__ref6 = (Parser_Lel_regular_expr_item*)&rhs[1]->user.regular_expr_item; +Parser_Lel_regular_expr_item *__ref7 = (Parser_Lel_regular_expr_item*)&rhs[1]->user.regular_expr_item; +Parser_Lel_regular_expr *__ref8 = (Parser_Lel_regular_expr*)&redLel->user.regular_expr; +Parser_Lel_regular_expr *__ref9 = (Parser_Lel_regular_expr*)&rhs[0]->user.regular_expr; +Parser_Lel_regular_expr *__ref10 = (Parser_Lel_regular_expr*)&redLel->user.regular_expr; +Parser_Lel_regular_expr *__ref11 = (Parser_Lel_regular_expr*)&rhs[0]->user.regular_expr; +Parser_Lel_regular_expr_item *__ref12 = (Parser_Lel_regular_expr_item*)&rhs[1]->user.regular_expr_item; +#line 1094 "rlparse.kl" + + /* An optimization to lessen the tree size. If a non-starred char is + * directly under the left side on the right and the right side is + * another non-starred char then paste them together and return the + * left side. Otherwise just put the two under a new reg exp node. */ + if ( (__ref0)->reItem->type == ReItem::Data && !(__ref1)->reItem->star && + (__ref2)->regExpr->type == RegExpr::RecurseItem && + (__ref3)->regExpr->item->type == ReItem::Data && !(__ref4)->regExpr->item->star ) + { + /* Append the right side to the right side of the left and toss the + * right side. */ + (__ref5)->regExpr->item->token.append( (__ref6)->reItem->token ); + delete (__ref7)->reItem; + (__ref8)->regExpr = (__ref9)->regExpr; + } + else { + (__ref10)->regExpr = new RegExpr( (__ref11)->regExpr, (__ref12)->reItem ); + } + + +#line 6023 "rlparse.cpp" +} break; +case 187: { +Parser_Lel_regular_expr *__ref0 = (Parser_Lel_regular_expr*)&redLel->user.regular_expr; +#line 1114 "rlparse.kl" + + /* Can't optimize the tree. */ + (__ref0)->regExpr = new RegExpr(); + + +#line 6033 "rlparse.cpp" +} break; +case 188: { +Parser_Lel_regular_expr_char *__ref0 = (Parser_Lel_regular_expr_char*)&rhs[0]->user.regular_expr_char; +Parser_Lel_regular_expr_item *__ref1 = (Parser_Lel_regular_expr_item*)&redLel->user.regular_expr_item; +Parser_Lel_regular_expr_char *__ref2 = (Parser_Lel_regular_expr_char*)&rhs[0]->user.regular_expr_char; +#line 1126 "rlparse.kl" + + (__ref0)->reItem->star = true; + (__ref1)->reItem = (__ref2)->reItem; + + +#line 6045 "rlparse.cpp" +} break; +case 189: { +Parser_Lel_regular_expr_item *__ref0 = (Parser_Lel_regular_expr_item*)&redLel->user.regular_expr_item; +Parser_Lel_regular_expr_char *__ref1 = (Parser_Lel_regular_expr_char*)&rhs[0]->user.regular_expr_char; +#line 1131 "rlparse.kl" + + (__ref0)->reItem = (__ref1)->reItem; + + +#line 6055 "rlparse.cpp" +} break; +case 190: { +Parser_Lel_regular_expr_char *__ref0 = (Parser_Lel_regular_expr_char*)&redLel->user.regular_expr_char; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_regular_expr_or_data *__ref2 = (Parser_Lel_regular_expr_or_data*)&rhs[1]->user.regular_expr_or_data; +#line 1143 "rlparse.kl" + + (__ref0)->reItem = new ReItem( (__ref1)->loc, (__ref2)->reOrBlock, ReItem::OrBlock ); + + +#line 6066 "rlparse.cpp" +} break; +case 191: { +Parser_Lel_regular_expr_char *__ref0 = (Parser_Lel_regular_expr_char*)&redLel->user.regular_expr_char; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_regular_expr_or_data *__ref2 = (Parser_Lel_regular_expr_or_data*)&rhs[1]->user.regular_expr_or_data; +#line 1147 "rlparse.kl" + + (__ref0)->reItem = new ReItem( (__ref1)->loc, (__ref2)->reOrBlock, ReItem::NegOrBlock ); + + +#line 6077 "rlparse.cpp" +} break; +case 192: { +Parser_Lel_regular_expr_char *__ref0 = (Parser_Lel_regular_expr_char*)&redLel->user.regular_expr_char; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1151 "rlparse.kl" + + (__ref0)->reItem = new ReItem( (__ref1)->loc, ReItem::Dot ); + + +#line 6087 "rlparse.cpp" +} break; +case 193: { +Parser_Lel_regular_expr_char *__ref0 = (Parser_Lel_regular_expr_char*)&redLel->user.regular_expr_char; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Token *__ref2 = (Token*)&rhs[0]->user.token; +#line 1155 "rlparse.kl" + + (__ref0)->reItem = new ReItem( (__ref1)->loc, *(__ref2) ); + + +#line 6098 "rlparse.cpp" +} break; +case 194: { +Parser_Lel_regular_expr_or_char *__ref0 = (Parser_Lel_regular_expr_or_char*)&rhs[1]->user.regular_expr_or_char; +Parser_Lel_regular_expr_or_data *__ref1 = (Parser_Lel_regular_expr_or_data*)&rhs[0]->user.regular_expr_or_data; +Parser_Lel_regular_expr_or_data *__ref2 = (Parser_Lel_regular_expr_or_data*)&rhs[0]->user.regular_expr_or_data; +Parser_Lel_regular_expr_or_data *__ref3 = (Parser_Lel_regular_expr_or_data*)&rhs[0]->user.regular_expr_or_data; +Parser_Lel_regular_expr_or_char *__ref4 = (Parser_Lel_regular_expr_or_char*)&rhs[1]->user.regular_expr_or_char; +Parser_Lel_regular_expr_or_char *__ref5 = (Parser_Lel_regular_expr_or_char*)&rhs[1]->user.regular_expr_or_char; +Parser_Lel_regular_expr_or_data *__ref6 = (Parser_Lel_regular_expr_or_data*)&redLel->user.regular_expr_or_data; +Parser_Lel_regular_expr_or_data *__ref7 = (Parser_Lel_regular_expr_or_data*)&rhs[0]->user.regular_expr_or_data; +Parser_Lel_regular_expr_or_data *__ref8 = (Parser_Lel_regular_expr_or_data*)&redLel->user.regular_expr_or_data; +Parser_Lel_regular_expr_or_data *__ref9 = (Parser_Lel_regular_expr_or_data*)&rhs[0]->user.regular_expr_or_data; +Parser_Lel_regular_expr_or_char *__ref10 = (Parser_Lel_regular_expr_or_char*)&rhs[1]->user.regular_expr_or_char; +#line 1167 "rlparse.kl" + + /* An optimization to lessen the tree size. If an or char is directly + * under the left side on the right and the right side is another or + * char then paste them together and return the left side. Otherwise + * just put the two under a new or data node. */ + if ( (__ref0)->reOrItem->type == ReOrItem::Data && + (__ref1)->reOrBlock->type == ReOrBlock::RecurseItem && + (__ref2)->reOrBlock->item->type == ReOrItem::Data ) + { + /* Append the right side to right side of the left and toss the + * right side. */ + (__ref3)->reOrBlock->item->token.append( (__ref4)->reOrItem->token ); + delete (__ref5)->reOrItem; + (__ref6)->reOrBlock = (__ref7)->reOrBlock; + } + else { + /* Can't optimize, put the left and right under a new node. */ + (__ref8)->reOrBlock = new ReOrBlock( (__ref9)->reOrBlock, (__ref10)->reOrItem ); + } + + +#line 6134 "rlparse.cpp" +} break; +case 195: { +Parser_Lel_regular_expr_or_data *__ref0 = (Parser_Lel_regular_expr_or_data*)&redLel->user.regular_expr_or_data; +#line 1188 "rlparse.kl" + + (__ref0)->reOrBlock = new ReOrBlock(); + + +#line 6143 "rlparse.cpp" +} break; +case 196: { +Parser_Lel_regular_expr_or_char *__ref0 = (Parser_Lel_regular_expr_or_char*)&redLel->user.regular_expr_or_char; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Token *__ref2 = (Token*)&rhs[0]->user.token; +#line 1200 "rlparse.kl" + + (__ref0)->reOrItem = new ReOrItem( (__ref1)->loc, *(__ref2) ); + + +#line 6154 "rlparse.cpp" +} break; +case 197: { +Parser_Lel_regular_expr_or_char *__ref0 = (Parser_Lel_regular_expr_or_char*)&redLel->user.regular_expr_or_char; +Token *__ref1 = (Token*)&rhs[1]->user.token; +Token *__ref2 = (Token*)&rhs[0]->user.token; +Token *__ref3 = (Token*)&rhs[2]->user.token; +#line 1204 "rlparse.kl" + + (__ref0)->reOrItem = new ReOrItem( (__ref1)->loc, (__ref2)->data[0], (__ref3)->data[0] ); + + +#line 6166 "rlparse.cpp" +} break; +case 198: { +Parser_Lel_inline_list *__ref0 = (Parser_Lel_inline_list*)&redLel->user.inline_list; +Parser_Lel_inline_list *__ref1 = (Parser_Lel_inline_list*)&rhs[0]->user.inline_list; +Parser_Lel_inline_list *__ref2 = (Parser_Lel_inline_list*)&redLel->user.inline_list; +Parser_Lel_inline_item *__ref3 = (Parser_Lel_inline_item*)&rhs[1]->user.inline_item; +#line 1221 "rlparse.kl" + + /* Append the item to the list, return the list. */ + (__ref0)->inlineList = (__ref1)->inlineList; + (__ref2)->inlineList->append( (__ref3)->inlineItem ); + + +#line 6180 "rlparse.cpp" +} break; +case 199: { +Parser_Lel_inline_list *__ref0 = (Parser_Lel_inline_list*)&redLel->user.inline_list; +#line 1228 "rlparse.kl" + + /* Start with empty list. */ + (__ref0)->inlineList = new InlineList; + + +#line 6190 "rlparse.cpp" +} break; +case 200: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +Parser_Lel_token_type *__ref2 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +#line 1243 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->token.loc, (__ref2)->token.data, InlineItem::Text ); + + +#line 6201 "rlparse.cpp" +} break; +case 201: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +Parser_Lel_token_type *__ref2 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +#line 1249 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->token.loc, (__ref2)->token.data, InlineItem::Text ); + + +#line 6212 "rlparse.cpp" +} break; +case 202: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_inline_item *__ref1 = (Parser_Lel_inline_item*)&rhs[0]->user.inline_item; +#line 1255 "rlparse.kl" + + /* Pass the inline item up. */ + (__ref0)->inlineItem = (__ref1)->inlineItem; + + +#line 6223 "rlparse.cpp" +} break; +case 203: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1262 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6231 "rlparse.cpp" +} break; +case 204: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1263 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6239 "rlparse.cpp" +} break; +case 205: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1264 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6247 "rlparse.cpp" +} break; +case 206: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1265 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6255 "rlparse.cpp" +} break; +case 207: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1266 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6263 "rlparse.cpp" +} break; +case 208: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1267 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6271 "rlparse.cpp" +} break; +case 209: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_inline_item *__ref1 = (Parser_Lel_inline_item*)&rhs[0]->user.inline_item; +#line 1271 "rlparse.kl" + + /* Pass up interpreted items of inline expressions. */ + (__ref0)->inlineItem = (__ref1)->inlineItem; + + +#line 6282 "rlparse.cpp" +} break; +case 210: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1276 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::Hold ); + + +#line 6292 "rlparse.cpp" +} break; +case 211: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_inline_item *__ref2 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_inline_list *__ref3 = (Parser_Lel_inline_list*)&rhs[1]->user.inline_list; +#line 1280 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::Exec ); + (__ref2)->inlineItem->children = (__ref3)->inlineList; + + +#line 6305 "rlparse.cpp" +} break; +case 212: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1285 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, + new NameRef(nameRef), InlineItem::Goto ); + + +#line 6316 "rlparse.cpp" +} break; +case 213: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_inline_item *__ref2 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_inline_list *__ref3 = (Parser_Lel_inline_list*)&rhs[2]->user.inline_list; +#line 1290 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::GotoExpr ); + (__ref2)->inlineItem->children = (__ref3)->inlineList; + + +#line 6329 "rlparse.cpp" +} break; +case 214: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1295 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, new NameRef(nameRef), InlineItem::Next ); + + +#line 6339 "rlparse.cpp" +} break; +case 215: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_inline_item *__ref2 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_inline_list *__ref3 = (Parser_Lel_inline_list*)&rhs[2]->user.inline_list; +#line 1299 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::NextExpr ); + (__ref2)->inlineItem->children = (__ref3)->inlineList; + + +#line 6352 "rlparse.cpp" +} break; +case 216: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1304 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, new NameRef(nameRef), InlineItem::Call ); + + +#line 6362 "rlparse.cpp" +} break; +case 217: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +Parser_Lel_inline_item *__ref2 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_inline_list *__ref3 = (Parser_Lel_inline_list*)&rhs[2]->user.inline_list; +#line 1308 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::CallExpr ); + (__ref2)->inlineItem->children = (__ref3)->inlineList; + + +#line 6375 "rlparse.cpp" +} break; +case 218: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1313 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::Ret ); + + +#line 6385 "rlparse.cpp" +} break; +case 219: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1317 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::Break ); + + +#line 6395 "rlparse.cpp" +} break; +case 220: { +Parser_Lel_inline_list *__ref0 = (Parser_Lel_inline_list*)&redLel->user.inline_list; +Parser_Lel_inline_list *__ref1 = (Parser_Lel_inline_list*)&rhs[0]->user.inline_list; +Parser_Lel_inline_list *__ref2 = (Parser_Lel_inline_list*)&redLel->user.inline_list; +Parser_Lel_inline_item *__ref3 = (Parser_Lel_inline_item*)&rhs[1]->user.inline_item; +#line 1325 "rlparse.kl" + + (__ref0)->inlineList = (__ref1)->inlineList; + (__ref2)->inlineList->append( (__ref3)->inlineItem ); + + +#line 6408 "rlparse.cpp" +} break; +case 221: { +Parser_Lel_inline_list *__ref0 = (Parser_Lel_inline_list*)&redLel->user.inline_list; +#line 1330 "rlparse.kl" + + /* Init the list used for this expr. */ + (__ref0)->inlineList = new InlineList; + + +#line 6418 "rlparse.cpp" +} break; +case 222: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +Parser_Lel_token_type *__ref2 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +#line 1339 "rlparse.kl" + + /* Return a text segment. */ + (__ref0)->inlineItem = new InlineItem( (__ref1)->token.loc, (__ref2)->token.data, InlineItem::Text ); + + +#line 6430 "rlparse.cpp" +} break; +case 223: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_token_type *__ref1 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +Parser_Lel_token_type *__ref2 = (Parser_Lel_token_type*)&rhs[0]->user.token_type; +#line 1345 "rlparse.kl" + + /* Return a text segment, must heap alloc the text. */ + (__ref0)->inlineItem = new InlineItem( (__ref1)->token.loc, (__ref2)->token.data, InlineItem::Text ); + + +#line 6442 "rlparse.cpp" +} break; +case 224: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Parser_Lel_inline_item *__ref1 = (Parser_Lel_inline_item*)&rhs[0]->user.inline_item; +#line 1351 "rlparse.kl" + + /* Pass the inline item up. */ + (__ref0)->inlineItem = (__ref1)->inlineItem; + + +#line 6453 "rlparse.cpp" +} break; +case 237: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1381 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::PChar ); + + +#line 6463 "rlparse.cpp" +} break; +case 238: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1386 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::Char ); + + +#line 6473 "rlparse.cpp" +} break; +case 239: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1391 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::Curs ); + + +#line 6483 "rlparse.cpp" +} break; +case 240: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1396 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, InlineItem::Targs ); + + +#line 6493 "rlparse.cpp" +} break; +case 241: { +Parser_Lel_inline_item *__ref0 = (Parser_Lel_inline_item*)&redLel->user.inline_item; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1401 "rlparse.kl" + + (__ref0)->inlineItem = new InlineItem( (__ref1)->loc, + new NameRef(nameRef), InlineItem::Entry ); + + +#line 6504 "rlparse.cpp" +} break; +case 243: { +#line 1412 "rlparse.kl" + + nameRef.empty(); + + +#line 6512 "rlparse.cpp" +} break; +case 245: { +#line 1422 "rlparse.kl" + + /* Insert an initial null pointer val to indicate the existence of the + * initial name seperator. */ + nameRef.setAs( 0 ); + + +#line 6522 "rlparse.cpp" +} break; +case 246: { +#line 1428 "rlparse.kl" + + nameRef.empty(); + + +#line 6530 "rlparse.cpp" +} break; +case 247: { +Token *__ref0 = (Token*)&rhs[2]->user.token; +#line 1435 "rlparse.kl" + + nameRef.append( (__ref0)->data ); + + +#line 6539 "rlparse.cpp" +} break; +case 248: { +Token *__ref0 = (Token*)&rhs[0]->user.token; +#line 1440 "rlparse.kl" + + nameRef.append( (__ref0)->data ); + + +#line 6548 "rlparse.cpp" +} break; +} + } + + if ( lel->child != 0 ) { + struct Parser_LangEl *first = lel->child; + struct Parser_LangEl *child = lel->child; + lel->child = 0; + while ( 1 ) { + if ( child->type < 226 ) { + } + else { + } + numNodes -= 1; + if ( child->next == 0 ) + break; + child = child->next; + } + child->next = pool; + pool = first; + } + } + +commit_base: + if ( sp > 0 ) { + sp -= 1; + if ( lel->retry == 0 ) { + lel = lel->prev; + goto commit_reverse; + } + else { + lel->retry = 0; + lel = lel->prev; + goto commit_upwards; + } + } + lel->retry = 0; + + lastFinal = lel; + numRetry = 0; + } + + if ( *action & 0x2 ) { + int reduction = *action >> 2; + struct Parser_LangEl *redLel; + + if ( input != 0 ) + input->causeReduce += 1; + + if ( pool == 0 ) { + if ( freshPos == 8128 ) { + struct Parser_Block* newBlock = (struct Parser_Block*) malloc( sizeof(struct Parser_Block) ); + newBlock->next = block; + block = newBlock; + freshEl = newBlock->data; + #ifdef KELBT_LOG_ACTIONS + cerr << "allocating 8128 LangEls" << endl; + #endif + freshPos = 0; + } + redLel = freshEl + freshPos++; + } + else { + redLel = pool; + pool = pool->next; + } + numNodes += 1; + + redLel->type = Parser_prodLhsIds[reduction]; + redLel->reduction = reduction; + redLel->child = 0; + redLel->next = 0; + redLel->retry = (lel->retry << 16); + redLel->causeReduce = 0; + lel->retry &= 0xffff0000; + + rhsLen = Parser_prodLengths[reduction]; + if ( rhsLen > 0 ) { + int r; + for ( r = rhsLen-1; r > 0; r-- ) { + rhs[r] = stackTop; + stackTop = stackTop->next; + } + rhs[0] = stackTop; + stackTop = stackTop->next; + rhs[0]->next = 0; + } +switch ( reduction ) { +case 225: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1358 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6643 "rlparse.cpp" +} break; +case 226: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1359 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6651 "rlparse.cpp" +} break; +case 227: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1360 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6659 "rlparse.cpp" +} break; +case 228: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1361 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6667 "rlparse.cpp" +} break; +case 229: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1362 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6675 "rlparse.cpp" +} break; +case 230: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1363 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6683 "rlparse.cpp" +} break; +case 231: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1364 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6691 "rlparse.cpp" +} break; +case 232: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1371 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6699 "rlparse.cpp" +} break; +case 233: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1372 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6707 "rlparse.cpp" +} break; +case 234: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1373 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6715 "rlparse.cpp" +} break; +case 235: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1374 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6723 "rlparse.cpp" +} break; +case 236: { +Parser_Lel_token_type *__ref0 = (Parser_Lel_token_type*)&redLel->user.token_type; +Token *__ref1 = (Token*)&rhs[0]->user.token; +#line 1375 "rlparse.kl" + (__ref0)->token = *(__ref1); + +#line 6731 "rlparse.cpp" +} break; +} + #ifdef KELBT_LOG_ACTIONS + cerr << "reduced: " + << Parser_prodNames[reduction] + << " rhsLen: " << rhsLen; + #endif + if ( action[1] == 0 ) + redLel->retry = 0; + else { + redLel->retry += 0x10000; + numRetry += 1; + #ifdef KELBT_LOG_ACTIONS + cerr << " retry: " << redLel; + #endif + } + + #ifdef KELBT_LOG_ACTIONS + cerr << endl; + #endif + + if ( rhsLen == 0 ) { + redLel->file = lel->file; + redLel->line = lel->line; + targState = curs; + } + else { + redLel->child = rhs[rhsLen-1]; + redLel->file = rhs[0]->file; + redLel->line = rhs[0]->line; + targState = rhs[0]->state; + } + + if ( induceReject ) { + #ifdef KELBT_LOG_ACTIONS + cerr << "error induced during reduction of " << + Parser_lelNames[redLel->type] << endl; + #endif + redLel->state = curs; + redLel->next = stackTop; + stackTop = redLel; + curs = targState; + goto parseError; + } + else { + redLel->next = input; + input = redLel; + } + } + + + curs = targState; + goto again; + +parseError: + #ifdef KELBT_LOG_BACKTRACK + cerr << "hit error" << endl; + #endif + if ( numRetry > 0 ) { + struct Parser_LangEl *redLel; + + if ( input != 0 ) { + redLel = input; + goto have_undo_element; + } + + while ( 1 ) { + redLel = stackTop; + if ( stackTop->type < 226 ) { + #ifdef KELBT_LOG_BACKTRACK + cerr << "backing up over terminal: " << + Parser_lelNames[stackTop->type] << endl; + #endif + stackTop = stackTop->next; + redLel->next = input; + input = redLel; + } + else { + #ifdef KELBT_LOG_BACKTRACK + cerr << "backing up over non-terminal: " << + Parser_lelNames[stackTop->type] << endl; + #endif + stackTop = stackTop->next; + struct Parser_LangEl *first = redLel->child; + if ( first == 0 ) + rhsLen = 0; + else { + rhsLen = 1; + while ( first->next != 0 ) { + first = first->next; + rhsLen += 1; + } + first->next = stackTop; + stackTop = redLel->child; + + struct Parser_LangEl *rhsEl = stackTop; + int p = rhsLen; + while ( p > 0 ) { + rhs[--p] = rhsEl; + rhsEl = rhsEl->next; + } + } + redLel->next = pool; + pool = redLel; + numNodes -= 1; + + if ( input != 0 ) + input->causeReduce -= 1; + } + +have_undo_element: + if ( redLel->retry == 0 ) { + if ( input != 0 && input->causeReduce == 0 ) { + #ifdef KELBT_LOG_BACKTRACK + cerr << "pushing back: " << Parser_lelNames[input->type] << endl; + #endif + input->next = queue; + queue = input; + input = 0; + } + } + else { + #ifdef KELBT_LOG_BACKTRACK + cerr << "found retry targ: " << redLel << endl; + #endif + numRetry -= 1; + #ifdef KELBT_LOG_BACKTRACK + cerr << "found retry: " << redLel << endl; + #endif + if ( redLel->retry & 0x0000ffff ) + curs = input->state; + else { + input->retry = redLel->retry >> 16; + if ( stackTop->state < 0 ) + curs = Parser_startState; + else { + curs = Parser_targs[(int)Parser_indicies[Parser_offsets[stackTop->state] + (stackTop->type - Parser_keys[stackTop->state<<1])]]; + } + } + goto again; + } + } + } + curs = -1; + errCount += 1; +_out: {} +#line 1459 "rlparse.kl" + return errCount == 0 ? 0 : -1; +} + +void Parser::tryMachineDef( InputLoc &loc, char *name, + MachineDef *machineDef, bool isInstance ) +{ + GraphDictEl *newEl = pd->graphDict.insert( name ); + if ( newEl != 0 ) { + /* New element in the dict, all good. */ + newEl->value = new VarDef( name, machineDef ); + newEl->isInstance = isInstance; + newEl->loc = loc; + newEl->value->isExport = exportContext[exportContext.length()-1]; + + /* It it is an instance, put on the instance list. */ + if ( isInstance ) + pd->instanceList.append( newEl ); + } + else { + // Recover by ignoring the duplicate. + error(loc) << "fsm \"" << name << "\" previously defined" << endl; + } +} + +ostream &Parser::parse_error( int tokId, Token &token ) +{ + /* Maintain the error count. */ + gblErrorCount += 1; + + cerr << token.loc << ": "; + cerr << "at token "; + if ( tokId < 128 ) + cerr << "\"" << Parser_lelNames[tokId] << "\""; + else + cerr << Parser_lelNames[tokId]; + if ( token.data != 0 ) + cerr << " with data \"" << token.data << "\""; + cerr << ": "; + + return cerr; +} + +int Parser::token( InputLoc &loc, int tokId, char *tokstart, int toklen ) +{ + Token token; + token.data = tokstart; + token.length = toklen; + token.loc = loc; + int res = parseLangEl( tokId, &token ); + if ( res < 0 ) { + parse_error(tokId, token) << "parse error" << endl; + exit(1); + } + return res; +} diff --git a/contrib/tools/ragel6/rlparse.h b/contrib/tools/ragel6/rlparse.h new file mode 100644 index 0000000000..e3f7c3c209 --- /dev/null +++ b/contrib/tools/ragel6/rlparse.h @@ -0,0 +1,210 @@ +/* Automatically generated by Kelbt from "rlparse.kh". + * + * Parts of this file are copied from Kelbt source covered by the GNU + * GPL. As a special exception, you may use the parts of this file copied + * from Kelbt source without restriction. The remainder is derived from + * "rlparse.kh" and inherits the copyright status of that file. + */ + +#line 1 "rlparse.kh" +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RLPARSE_H +#define _RLPARSE_H + +#include <iostream> +#include "avltree.h" +#include "parsedata.h" + + +/* Import scanner tokens. */ +#define IMP_Word 128 +#define IMP_Literal 129 +#define IMP_UInt 130 +#define IMP_Define 131 + +/* This is used for tracking the include files/machine pairs. */ +struct IncludeHistoryItem +{ + IncludeHistoryItem( const char *fileName, const char *sectionName ) + : fileName(fileName), sectionName(sectionName) {} + + const char *fileName; + const char *sectionName; +}; + +typedef Vector<IncludeHistoryItem> IncludeHistory; + +struct Parser +{ +#line 102 "rlparse.kh" + + + #line 63 "rlparse.h" + struct Parser_Block *block; + struct Parser_LangEl *freshEl; + int freshPos; + struct Parser_LangEl *pool; + int numRetry; + int numNodes; + struct Parser_LangEl *stackTop; + struct Parser_LangEl *lastFinal; + int errCount; + int curs; +#line 105 "rlparse.kh" + + void init(); + int parseLangEl( int type, const Token *token ); + + Parser( const char *fileName, char *sectionName, InputLoc §ionLoc ) + : sectionName(sectionName) + { + pd = new ParseData( fileName, sectionName, sectionLoc ); + exportContext.append( false ); + includeHistory.append( IncludeHistoryItem( + fileName, sectionName ) ); + } + + int token( InputLoc &loc, int tokId, char *tokstart, int toklen ); + void tryMachineDef( InputLoc &loc, char *name, + MachineDef *machineDef, bool isInstance ); + + /* Report an error encountered by the parser. */ + ostream &parse_error( int tokId, Token &token ); + + ParseData *pd; + + /* The name of the root section, this does not change during an include. */ + char *sectionName; + + NameRef nameRef; + NameRefList nameRefList; + + Vector<bool> exportContext; + IncludeHistory includeHistory; + + Parser *prev, *next; +}; + +#line 109 "rlparse.h" +#define TK_Word 128 +#define TK_Literal 129 +#define TK_Number 130 +#define TK_EndSection 131 +#define TK_UInt 132 +#define TK_Hex 133 +#define TK_DotDot 134 +#define TK_ColonGt 135 +#define TK_ColonGtGt 136 +#define TK_LtColon 137 +#define TK_Arrow 138 +#define TK_DoubleArrow 139 +#define TK_StarStar 140 +#define TK_ColonEquals 141 +#define TK_NameSep 142 +#define TK_BarStar 143 +#define TK_DashDash 144 +#define TK_StartCond 145 +#define TK_AllCond 146 +#define TK_LeavingCond 147 +#define TK_Middle 148 +#define TK_StartGblError 149 +#define TK_AllGblError 150 +#define TK_FinalGblError 151 +#define TK_NotFinalGblError 152 +#define TK_NotStartGblError 153 +#define TK_MiddleGblError 154 +#define TK_StartLocalError 155 +#define TK_AllLocalError 156 +#define TK_FinalLocalError 157 +#define TK_NotFinalLocalError 158 +#define TK_NotStartLocalError 159 +#define TK_MiddleLocalError 160 +#define TK_StartEOF 161 +#define TK_AllEOF 162 +#define TK_FinalEOF 163 +#define TK_NotFinalEOF 164 +#define TK_NotStartEOF 165 +#define TK_MiddleEOF 166 +#define TK_StartToState 167 +#define TK_AllToState 168 +#define TK_FinalToState 169 +#define TK_NotFinalToState 170 +#define TK_NotStartToState 171 +#define TK_MiddleToState 172 +#define TK_StartFromState 173 +#define TK_AllFromState 174 +#define TK_FinalFromState 175 +#define TK_NotFinalFromState 176 +#define TK_NotStartFromState 177 +#define TK_MiddleFromState 178 +#define RE_Slash 179 +#define RE_SqOpen 180 +#define RE_SqOpenNeg 181 +#define RE_SqClose 182 +#define RE_Dot 183 +#define RE_Star 184 +#define RE_Dash 185 +#define RE_Char 186 +#define IL_WhiteSpace 187 +#define IL_Comment 188 +#define IL_Literal 189 +#define IL_Symbol 190 +#define KW_Machine 191 +#define KW_Include 192 +#define KW_Import 193 +#define KW_Write 194 +#define KW_Action 195 +#define KW_AlphType 196 +#define KW_Range 197 +#define KW_GetKey 198 +#define KW_InWhen 199 +#define KW_When 200 +#define KW_OutWhen 201 +#define KW_Eof 202 +#define KW_Err 203 +#define KW_Lerr 204 +#define KW_To 205 +#define KW_From 206 +#define KW_Export 207 +#define KW_PrePush 208 +#define KW_PostPop 209 +#define KW_Length 210 +#define KW_Break 211 +#define KW_Exec 212 +#define KW_Hold 213 +#define KW_PChar 214 +#define KW_Char 215 +#define KW_Goto 216 +#define KW_Call 217 +#define KW_Ret 218 +#define KW_CurState 219 +#define KW_TargState 220 +#define KW_Entry 221 +#define KW_Next 222 +#define KW_Variable 223 +#define KW_Access 224 +#define Parser_tk_eof 225 + +#line 140 "rlparse.kh" + +#endif diff --git a/contrib/tools/ragel6/rlparse.kh b/contrib/tools/ragel6/rlparse.kh new file mode 100644 index 0000000000..899bbbcee2 --- /dev/null +++ b/contrib/tools/ragel6/rlparse.kh @@ -0,0 +1,141 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RLPARSE_H +#define _RLPARSE_H + +#include <iostream> +#include "avltree.h" +#include "parsedata.h" + + +/* Import scanner tokens. */ +#define IMP_Word 128 +#define IMP_Literal 129 +#define IMP_UInt 130 +#define IMP_Define 131 + +/* This is used for tracking the include files/machine pairs. */ +struct IncludeHistoryItem +{ + IncludeHistoryItem( const char *fileName, const char *sectionName ) + : fileName(fileName), sectionName(sectionName) {} + + const char *fileName; + const char *sectionName; +}; + +typedef Vector<IncludeHistoryItem> IncludeHistory; + +struct Parser +{ +%%{ + parser Parser; + + # General tokens. + token TK_Word, TK_Literal, TK_Number, TK_EndSection, TK_UInt, TK_Hex, + TK_Word, TK_Literal, TK_DotDot, TK_ColonGt, TK_ColonGtGt, TK_LtColon, + TK_Arrow, TK_DoubleArrow, TK_StarStar, TK_ColonEquals, TK_NameSep, + TK_BarStar, TK_DashDash; + + # Conditions. + token TK_StartCond, TK_AllCond, TK_LeavingCond; + + # State embedding actions. + token TK_Middle; + + # Global error actions. + token TK_StartGblError, TK_AllGblError, TK_FinalGblError, + TK_NotFinalGblError, TK_NotStartGblError, TK_MiddleGblError; + + # Local error actions. + token TK_StartLocalError, TK_AllLocalError, TK_FinalLocalError, + TK_NotFinalLocalError, TK_NotStartLocalError, TK_MiddleLocalError; + + # EOF Action embedding. + token TK_StartEOF, TK_AllEOF, TK_FinalEOF, TK_NotFinalEOF, TK_NotStartEOF, + TK_MiddleEOF; + + # To State Actions. + token TK_StartToState, TK_AllToState, TK_FinalToState, TK_NotFinalToState, + TK_NotStartToState, TK_MiddleToState; + + # In State Actions. + token TK_StartFromState, TK_AllFromState, TK_FinalFromState, + TK_NotFinalFromState, TK_NotStartFromState, TK_MiddleFromState; + + # Regular expression tokens. */ + token RE_Slash, RE_SqOpen, RE_SqOpenNeg, RE_SqClose, RE_Dot, RE_Star, + RE_Dash, RE_Char; + + # Tokens specific to inline code. + token IL_WhiteSpace, IL_Comment, IL_Literal, IL_Symbol; + + # Keywords. + token KW_Machine, KW_Include, KW_Import, KW_Write, KW_Action, KW_AlphType, + KW_Range, KW_GetKey, KW_Include, KW_Write, KW_Machine, KW_InWhen, + KW_When, KW_OutWhen, KW_Eof, KW_Err, KW_Lerr, KW_To, KW_From, + KW_Export, KW_PrePush, KW_PostPop, KW_Length; + + # Specials in code blocks. + token KW_Break, KW_Exec, KW_Hold, KW_PChar, KW_Char, KW_Goto, KW_Call, + KW_Ret, KW_CurState, KW_TargState, KW_Entry, KW_Next, KW_Exec, + KW_Variable, KW_Access; +}%% + + %% write instance_data; + + void init(); + int parseLangEl( int type, const Token *token ); + + Parser( const char *fileName, char *sectionName, InputLoc §ionLoc ) + : sectionName(sectionName) + { + pd = new ParseData( fileName, sectionName, sectionLoc ); + exportContext.append( false ); + includeHistory.append( IncludeHistoryItem( + fileName, sectionName ) ); + } + + int token( InputLoc &loc, int tokId, char *tokstart, int toklen ); + void tryMachineDef( InputLoc &loc, char *name, + MachineDef *machineDef, bool isInstance ); + + /* Report an error encountered by the parser. */ + ostream &parse_error( int tokId, Token &token ); + + ParseData *pd; + + /* The name of the root section, this does not change during an include. */ + char *sectionName; + + NameRef nameRef; + NameRefList nameRefList; + + Vector<bool> exportContext; + IncludeHistory includeHistory; + + Parser *prev, *next; +}; + +%% write token_defs; + +#endif diff --git a/contrib/tools/ragel6/rlparse.kl b/contrib/tools/ragel6/rlparse.kl new file mode 100644 index 0000000000..36b8777b84 --- /dev/null +++ b/contrib/tools/ragel6/rlparse.kl @@ -0,0 +1,1513 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "rlparse.h" +#include "ragel.h" +#include <iostream> +#include <errno.h> +#include <stdlib.h> + +using std::cout; +using std::cerr; +using std::endl; + +%%{ + +parser Parser; + +include "rlparse.kh"; + +start: section_list; + +section_list: section_list statement_list TK_EndSection; +section_list: ; + +statement_list: statement_list statement; +statement_list: ; + +statement: assignment commit; +statement: instantiation commit; +statement: action_spec commit; +statement: alphtype_spec commit; +statement: range_spec commit; +statement: getkey_spec commit; +statement: access_spec commit; +statement: variable_spec commit; +statement: export_block commit; +statement: pre_push_spec commit; +statement: post_pop_spec commit; +statement: length_spec commit; + +length_spec: + KW_Length TK_Word ';' + final { + LengthDef *lengthDef = new LengthDef( $2->data ); + pd->lengthDefList.append( lengthDef ); + + /* Generic creation of machine for instantiation and assignment. */ + MachineDef *machineDef = new MachineDef( lengthDef ); + tryMachineDef( $2->loc, $2->data, machineDef, false ); + }; + +pre_push_spec: + KW_PrePush '{' inline_block '}' + final { + if ( pd->prePushExpr != 0 ) { + /* Recover by just ignoring the duplicate. */ + error($2->loc) << "pre_push code already defined" << endl; + } + + pd->prePushExpr = $3->inlineList; + }; + + +post_pop_spec: + KW_PostPop '{' inline_block '}' + final { + if ( pd->postPopExpr != 0 ) { + /* Recover by just ignoring the duplicate. */ + error($2->loc) << "post_pop code already defined" << endl; + } + + pd->postPopExpr = $3->inlineList; + }; + + +export_open: KW_Export + final { + exportContext.append( true ); + }; + +nonterm opt_export +{ + bool isSet; +}; + +opt_export: export_open final { $$->isSet = true; }; +opt_export: final { $$->isSet = false; }; + +export_block: export_open '{' statement_list '}' + final { + exportContext.remove( exportContext.length()-1 ); + }; + +assignment: + opt_export machine_name '=' join ';' final { + /* Main machine must be an instance. */ + bool isInstance = false; + if ( strcmp($2->token.data, mainMachine) == 0 ) { + warning($2->token.loc) << + "main machine will be implicitly instantiated" << endl; + isInstance = true; + } + + /* Generic creation of machine for instantiation and assignment. */ + MachineDef *machineDef = new MachineDef( $4->join ); + tryMachineDef( $2->token.loc, $2->token.data, machineDef, isInstance ); + + if ( $1->isSet ) + exportContext.remove( exportContext.length()-1 ); + + $4->join->loc = $3->loc; + }; + +instantiation: + opt_export machine_name TK_ColonEquals join_or_lm ';' final { + /* Generic creation of machine for instantiation and assignment. */ + tryMachineDef( $2->token.loc, $2->token.data, $4->machineDef, true ); + + if ( $1->isSet ) + exportContext.remove( exportContext.length()-1 ); + + /* Pass a location to join_or_lm */ + if ( $4->machineDef->join != 0 ) + $4->machineDef->join->loc = $3->loc; + }; + +type token_type +{ + Token token; +}; + +nonterm machine_name uses token_type; + +machine_name: + TK_Word final { + /* Make/get the priority key. The name may have already been referenced + * and therefore exist. */ + PriorDictEl *priorDictEl; + if ( pd->priorDict.insert( $1->data, pd->nextPriorKey, &priorDictEl ) ) + pd->nextPriorKey += 1; + pd->curDefPriorKey = priorDictEl->value; + + /* Make/get the local error key. */ + LocalErrDictEl *localErrDictEl; + if ( pd->localErrDict.insert( $1->data, pd->nextLocalErrKey, &localErrDictEl ) ) + pd->nextLocalErrKey += 1; + pd->curDefLocalErrKey = localErrDictEl->value; + + $$->token = *$1; + }; + +action_spec: + KW_Action TK_Word '{' inline_block '}' final { + if ( pd->actionDict.find( $2->data ) ) { + /* Recover by just ignoring the duplicate. */ + error($2->loc) << "action \"" << $2->data << "\" already defined" << endl; + } + else { + //cerr << "NEW ACTION " << $2->data << " " << $4->inlineList << endl; + /* Add the action to the list of actions. */ + Action *newAction = new Action( $3->loc, $2->data, + $4->inlineList, pd->nextCondId++ ); + + /* Insert to list and dict. */ + pd->actionList.append( newAction ); + pd->actionDict.insert( newAction ); + } + }; + +# Specifies the data type of the input alphabet. One or two words followed by a +# semi-colon. +alphtype_spec: + KW_AlphType TK_Word TK_Word ';' final { + if ( ! pd->setAlphType( $1->loc, $2->data, $3->data ) ) { + // Recover by ignoring the alphtype statement. + error($2->loc) << "\"" << $2->data << + " " << $3->data << "\" is not a valid alphabet type" << endl; + } + }; + +alphtype_spec: + KW_AlphType TK_Word ';' final { + if ( ! pd->setAlphType( $1->loc, $2->data ) ) { + // Recover by ignoring the alphtype statement. + error($2->loc) << "\"" << $2->data << + "\" is not a valid alphabet type" << endl; + } + }; + +# Specifies a range to assume that the input characters will fall into. +range_spec: + KW_Range alphabet_num alphabet_num ';' final { + // Save the upper and lower ends of the range and emit the line number. + pd->lowerNum = $2->token.data; + pd->upperNum = $3->token.data; + pd->rangeLowLoc = $2->token.loc; + pd->rangeHighLoc = $3->token.loc; + }; + +getkey_spec: + KW_GetKey inline_expr ';' final { + pd->getKeyExpr = $2->inlineList; + }; + +access_spec: + KW_Access inline_expr ';' final { + pd->accessExpr = $2->inlineList; + }; + +variable_spec: + KW_Variable opt_whitespace TK_Word inline_expr ';' final { + /* FIXME: Need to implement the rest of this. */ + bool wasSet = pd->setVariable( $3->data, $4->inlineList ); + if ( !wasSet ) + error($3->loc) << "bad variable name" << endl; + }; + +opt_whitespace: opt_whitespace IL_WhiteSpace; +opt_whitespace: ; + +# +# Expressions +# + +nonterm join_or_lm +{ + MachineDef *machineDef; +}; + +join_or_lm: + join final { + $$->machineDef = new MachineDef( $1->join ); + }; +join_or_lm: + TK_BarStar lm_part_list '*' '|' final { + /* Create a new factor going to a longest match structure. Record + * in the parse data that we have a longest match. */ + LongestMatch *lm = new LongestMatch( $1->loc, $2->lmPartList ); + pd->lmList.append( lm ); + for ( LmPartList::Iter lmp = *($2->lmPartList); lmp.lte(); lmp++ ) + lmp->longestMatch = lm; + $$->machineDef = new MachineDef( lm ); + }; + +nonterm lm_part_list +{ + LmPartList *lmPartList; +}; + +lm_part_list: + lm_part_list longest_match_part + final { + if ( $2->lmPart != 0 ) + $1->lmPartList->append( $2->lmPart ); + $$->lmPartList = $1->lmPartList; + }; +lm_part_list: + longest_match_part + final { + /* Create a new list with the part. */ + $$->lmPartList = new LmPartList; + if ( $1->lmPart != 0 ) + $$->lmPartList->append( $1->lmPart ); + }; + +nonterm longest_match_part +{ + LongestMatchPart *lmPart; +}; + +longest_match_part: + action_spec final { $$->lmPart = 0; }; +longest_match_part: + assignment final { $$->lmPart = 0; }; +longest_match_part: + join opt_lm_part_action ';' final { + $$->lmPart = 0; + Action *action = $2->action; + if ( action != 0 ) + action->isLmAction = true; + $$->lmPart = new LongestMatchPart( $1->join, action, + $3->loc, pd->nextLongestMatchId++ ); + + /* Provide a location to join. Unfortunately We don't + * have the start of the join as in other occurances. Use the end. */ + $1->join->loc = $3->loc; + }; + +nonterm opt_lm_part_action +{ + Action *action; +}; + +opt_lm_part_action: + TK_DoubleArrow action_embed final { + $$->action = $2->action; + }; +opt_lm_part_action: + action_embed_block final { + $$->action = $1->action; + }; +opt_lm_part_action: + final { + $$->action = 0; + }; + + +nonterm join +{ + Join *join; +}; + +join: + join ',' expression final { + /* Append the expression to the list and return it. */ + $1->join->exprList.append( $3->expression ); + $$->join = $1->join; + }; +join: + expression final { + $$->join = new Join( $1->expression ); + }; + +nonterm expression +{ + Expression *expression; +}; + +expression: + expression '|' term_short final { + $$->expression = new Expression( $1->expression, + $3->term, Expression::OrType ); + }; +expression: + expression '&' term_short final { + $$->expression = new Expression( $1->expression, + $3->term, Expression::IntersectType ); + }; +expression: + expression '-' term_short final { + $$->expression = new Expression( $1->expression, + $3->term, Expression::SubtractType ); + }; +expression: + expression TK_DashDash term_short final { + $$->expression = new Expression( $1->expression, + $3->term, Expression::StrongSubtractType ); + }; +expression: + term_short final { + $$->expression = new Expression( $1->term ); + }; + +# This is where we resolve the ambiguity involving -. By default ragel tries to +# do a longest match, which gives precedence to a concatenation because it is +# innermost. What we need is to force term into a shortest match so that when - +# is seen it doesn't try to extend term with a concatenation, but ends term and +# goes for a subtraction. +# +# The shortest tag overrides the default longest match action ordering strategy +# and instead forces a shortest match stragegy. The wrap the term production in +# a new nonterminal 'term_short' to guarantee the shortest match behaviour. + +shortest term_short; +nonterm term_short +{ + Term *term; +}; + +term_short: + term final { + $$->term = $1->term; + }; + +nonterm term +{ + Term *term; +}; + +term: + term factor_with_label final { + $$->term = new Term( $1->term, $2->factorWithAug ); + }; +term: + term '.' factor_with_label final { + $$->term = new Term( $1->term, $3->factorWithAug ); + }; +term: + term TK_ColonGt factor_with_label final { + $$->term = new Term( $1->term, $3->factorWithAug, Term::RightStartType ); + }; +term: + term TK_ColonGtGt factor_with_label final { + $$->term = new Term( $1->term, $3->factorWithAug, Term::RightFinishType ); + }; +term: + term TK_LtColon factor_with_label final { + $$->term = new Term( $1->term, + $3->factorWithAug, Term::LeftType ); + }; +term: + factor_with_label final { + $$->term = new Term( $1->factorWithAug ); + }; + +nonterm factor_with_label +{ + FactorWithAug *factorWithAug; +}; + +factor_with_label: + TK_Word ':' factor_with_label final { + /* Add the label to the list and pass the factor up. */ + $3->factorWithAug->labels.prepend( Label($1->loc, $1->data) ); + $$->factorWithAug = $3->factorWithAug; + }; +factor_with_label: + factor_with_ep final { + $$->factorWithAug = $1->factorWithAug; + }; + +nonterm factor_with_ep +{ + FactorWithAug *factorWithAug; +}; + +factor_with_ep: + factor_with_ep TK_Arrow local_state_ref final { + /* Add the target to the list and return the factor object. */ + $1->factorWithAug->epsilonLinks.append( EpsilonLink( $2->loc, nameRef ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_ep: + factor_with_aug final { + $$->factorWithAug = $1->factorWithAug; + }; + +nonterm factor_with_aug +{ + FactorWithAug *factorWithAug; +}; + +factor_with_aug: + factor_with_aug aug_type_base action_embed final { + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + $1->factorWithAug->actions.append( + ParserAction( $2->loc, $2->augType, 0, $3->action ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_base priority_aug final { + /* Append the named priority to the factorWithAug and pass it up. */ + $1->factorWithAug->priorityAugs.append( + PriorityAug( $2->augType, pd->curDefPriorKey, $3->priorityNum ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_base '(' priority_name ',' priority_aug ')' final { + /* Append the priority using a default name. */ + $1->factorWithAug->priorityAugs.append( + PriorityAug( $2->augType, $4->priorityName, $6->priorityNum ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_cond action_embed final { + $1->factorWithAug->conditions.append( ConditionTest( $2->loc, + $2->augType, $3->action, true ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_cond '!' action_embed final { + $1->factorWithAug->conditions.append( ConditionTest( $2->loc, + $2->augType, $4->action, false ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_to_state action_embed final { + /* Append the action, pass it up. */ + $1->factorWithAug->actions.append( ParserAction( $2->loc, + $2->augType, 0, $3->action ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_from_state action_embed final { + /* Append the action, pass it up. */ + $1->factorWithAug->actions.append( ParserAction( $2->loc, + $2->augType, 0, $3->action ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_eof action_embed final { + /* Append the action, pass it up. */ + $1->factorWithAug->actions.append( ParserAction( $2->loc, + $2->augType, 0, $3->action ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_gbl_error action_embed final { + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + $1->factorWithAug->actions.append( ParserAction( $2->loc, + $2->augType, pd->curDefLocalErrKey, $3->action ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_local_error action_embed final { + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + $1->factorWithAug->actions.append( ParserAction( $2->loc, + $2->augType, pd->curDefLocalErrKey, $3->action ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_aug aug_type_local_error '(' local_err_name ',' action_embed ')' final { + /* Append the action to the factorWithAug, record the refernce from + * factorWithAug to the action and pass up the factorWithAug. */ + $1->factorWithAug->actions.append( ParserAction( $2->loc, + $2->augType, $4->error_name, $6->action ) ); + $$->factorWithAug = $1->factorWithAug; + }; +factor_with_aug: + factor_with_rep final { + $$->factorWithAug = new FactorWithAug( $1->factorWithRep ); + }; + +type aug_type +{ + InputLoc loc; + AugType augType; +}; + +# Classes of transtions on which to embed actions or change priorities. +nonterm aug_type_base uses aug_type; + +aug_type_base: '@' final { $$->loc = $1->loc; $$->augType = at_finish; }; +aug_type_base: '%' final { $$->loc = $1->loc; $$->augType = at_leave; }; +aug_type_base: '$' final { $$->loc = $1->loc; $$->augType = at_all; }; +aug_type_base: '>' final { $$->loc = $1->loc; $$->augType = at_start; }; + +# Embedding conditions. +nonterm aug_type_cond uses aug_type; + +aug_type_cond: TK_StartCond final { $$->loc = $1->loc; $$->augType = at_start; }; +aug_type_cond: '>' KW_When final { $$->loc = $1->loc; $$->augType = at_start; }; +aug_type_cond: TK_AllCond final { $$->loc = $1->loc; $$->augType = at_all; }; +aug_type_cond: '$' KW_When final { $$->loc = $1->loc; $$->augType = at_all; }; +aug_type_cond: TK_LeavingCond final { $$->loc = $1->loc; $$->augType = at_leave; }; +aug_type_cond: '%' KW_When final { $$->loc = $1->loc; $$->augType = at_leave; }; +aug_type_cond: KW_When final { $$->loc = $1->loc; $$->augType = at_all; }; +aug_type_cond: KW_InWhen final { $$->loc = $1->loc; $$->augType = at_start; }; +aug_type_cond: KW_OutWhen final { $$->loc = $1->loc; $$->augType = at_leave; }; + +# +# To state actions. +# + +nonterm aug_type_to_state uses aug_type; + +aug_type_to_state: TK_StartToState + final { $$->loc = $1->loc; $$->augType = at_start_to_state; }; +aug_type_to_state: '>' KW_To + final { $$->loc = $1->loc; $$->augType = at_start_to_state; }; + +aug_type_to_state: TK_NotStartToState + final { $$->loc = $1->loc; $$->augType = at_not_start_to_state; }; +aug_type_to_state: '<' KW_To + final { $$->loc = $1->loc; $$->augType = at_not_start_to_state; }; + +aug_type_to_state: TK_AllToState + final { $$->loc = $1->loc; $$->augType = at_all_to_state; }; +aug_type_to_state: '$' KW_To + final { $$->loc = $1->loc; $$->augType = at_all_to_state; }; + +aug_type_to_state: TK_FinalToState + final { $$->loc = $1->loc; $$->augType = at_final_to_state; }; +aug_type_to_state: '%' KW_To + final { $$->loc = $1->loc; $$->augType = at_final_to_state; }; + +aug_type_to_state: TK_NotFinalToState + final { $$->loc = $1->loc; $$->augType = at_not_final_to_state; }; +aug_type_to_state: '@' KW_To + final { $$->loc = $1->loc; $$->augType = at_not_final_to_state; }; + +aug_type_to_state: TK_MiddleToState + final { $$->loc = $1->loc; $$->augType = at_middle_to_state; }; +aug_type_to_state: TK_Middle KW_To + final { $$->loc = $1->loc; $$->augType = at_middle_to_state; }; + +# +# From state actions. +# + +nonterm aug_type_from_state uses aug_type; + +aug_type_from_state: TK_StartFromState + final { $$->loc = $1->loc; $$->augType = at_start_from_state; }; +aug_type_from_state: '>' KW_From + final { $$->loc = $1->loc; $$->augType = at_start_from_state; }; + +aug_type_from_state: TK_NotStartFromState + final { $$->loc = $1->loc; $$->augType = at_not_start_from_state; }; +aug_type_from_state: '<' KW_From + final { $$->loc = $1->loc; $$->augType = at_not_start_from_state; }; + +aug_type_from_state: TK_AllFromState + final { $$->loc = $1->loc; $$->augType = at_all_from_state; }; +aug_type_from_state: '$' KW_From + final { $$->loc = $1->loc; $$->augType = at_all_from_state; }; + +aug_type_from_state: TK_FinalFromState + final { $$->loc = $1->loc; $$->augType = at_final_from_state; }; +aug_type_from_state: '%' KW_From + final { $$->loc = $1->loc; $$->augType = at_final_from_state; }; + +aug_type_from_state: TK_NotFinalFromState + final { $$->loc = $1->loc; $$->augType = at_not_final_from_state; }; +aug_type_from_state: '@' KW_From + final { $$->loc = $1->loc; $$->augType = at_not_final_from_state; }; + +aug_type_from_state: TK_MiddleFromState + final { $$->loc = $1->loc; $$->augType = at_middle_from_state; }; +aug_type_from_state: TK_Middle KW_From + final { $$->loc = $1->loc; $$->augType = at_middle_from_state; }; + +# +# Eof state actions. +# + +nonterm aug_type_eof uses aug_type; + +aug_type_eof: TK_StartEOF + final { $$->loc = $1->loc; $$->augType = at_start_eof; }; +aug_type_eof: '>' KW_Eof + final { $$->loc = $1->loc; $$->augType = at_start_eof; }; + +aug_type_eof: TK_NotStartEOF + final { $$->loc = $1->loc; $$->augType = at_not_start_eof; }; +aug_type_eof: '<' KW_Eof + final { $$->loc = $1->loc; $$->augType = at_not_start_eof; }; + +aug_type_eof: TK_AllEOF + final { $$->loc = $1->loc; $$->augType = at_all_eof; }; +aug_type_eof: '$' KW_Eof + final { $$->loc = $1->loc; $$->augType = at_all_eof; }; + +aug_type_eof: TK_FinalEOF + final { $$->loc = $1->loc; $$->augType = at_final_eof; }; +aug_type_eof: '%' KW_Eof + final { $$->loc = $1->loc; $$->augType = at_final_eof; }; + +aug_type_eof: TK_NotFinalEOF + final { $$->loc = $1->loc; $$->augType = at_not_final_eof; }; +aug_type_eof: '@' KW_Eof + final { $$->loc = $1->loc; $$->augType = at_not_final_eof; }; + +aug_type_eof: TK_MiddleEOF + final { $$->loc = $1->loc; $$->augType = at_middle_eof; }; +aug_type_eof: TK_Middle KW_Eof + final { $$->loc = $1->loc; $$->augType = at_middle_eof; }; + +# +# Global error actions. +# + +nonterm aug_type_gbl_error uses aug_type; + +aug_type_gbl_error: TK_StartGblError + final { $$->loc = $1->loc; $$->augType = at_start_gbl_error; }; +aug_type_gbl_error: '>' KW_Err + final { $$->loc = $1->loc; $$->augType = at_start_gbl_error; }; + +aug_type_gbl_error: TK_NotStartGblError + final { $$->loc = $1->loc; $$->augType = at_not_start_gbl_error; }; +aug_type_gbl_error: '<' KW_Err + final { $$->loc = $1->loc; $$->augType = at_not_start_gbl_error; }; + +aug_type_gbl_error: TK_AllGblError + final { $$->loc = $1->loc; $$->augType = at_all_gbl_error; }; +aug_type_gbl_error: '$' KW_Err + final { $$->loc = $1->loc; $$->augType = at_all_gbl_error; }; + +aug_type_gbl_error: TK_FinalGblError + final { $$->loc = $1->loc; $$->augType = at_final_gbl_error; }; +aug_type_gbl_error: '%' KW_Err + final { $$->loc = $1->loc; $$->augType = at_final_gbl_error; }; + +aug_type_gbl_error: TK_NotFinalGblError + final { $$->loc = $1->loc; $$->augType = at_not_final_gbl_error; }; +aug_type_gbl_error: '@' KW_Err + final { $$->loc = $1->loc; $$->augType = at_not_final_gbl_error; }; + +aug_type_gbl_error: TK_MiddleGblError + final { $$->loc = $1->loc; $$->augType = at_middle_gbl_error; }; +aug_type_gbl_error: TK_Middle KW_Err + final { $$->loc = $1->loc; $$->augType = at_middle_gbl_error; }; + + +# +# Local error actions. +# + +nonterm aug_type_local_error uses aug_type; + +aug_type_local_error: TK_StartLocalError + final { $$->loc = $1->loc; $$->augType = at_start_local_error; }; +aug_type_local_error: '>' KW_Lerr + final { $$->loc = $1->loc; $$->augType = at_start_local_error; }; + +aug_type_local_error: TK_NotStartLocalError + final { $$->loc = $1->loc; $$->augType = at_not_start_local_error; }; +aug_type_local_error: '<' KW_Lerr + final { $$->loc = $1->loc; $$->augType = at_not_start_local_error; }; + +aug_type_local_error: TK_AllLocalError + final { $$->loc = $1->loc; $$->augType = at_all_local_error; }; +aug_type_local_error: '$' KW_Lerr + final { $$->loc = $1->loc; $$->augType = at_all_local_error; }; + +aug_type_local_error: TK_FinalLocalError + final { $$->loc = $1->loc; $$->augType = at_final_local_error; }; +aug_type_local_error: '%' KW_Lerr + final { $$->loc = $1->loc; $$->augType = at_final_local_error; }; + +aug_type_local_error: TK_NotFinalLocalError + final { $$->loc = $1->loc; $$->augType = at_not_final_local_error; }; +aug_type_local_error: '@' KW_Lerr + final { $$->loc = $1->loc; $$->augType = at_not_final_local_error; }; + +aug_type_local_error: TK_MiddleLocalError + final { $$->loc = $1->loc; $$->augType = at_middle_local_error; }; +aug_type_local_error: TK_Middle KW_Lerr + final { $$->loc = $1->loc; $$->augType = at_middle_local_error; }; + + +type action_ref +{ + Action *action; +}; + +# Different ways to embed actions. A TK_Word is reference to an action given by +# the user as a statement in the fsm specification. An action can also be +# specified immediately. +nonterm action_embed uses action_ref; + +action_embed: action_embed_word final { $$->action = $1->action; }; +action_embed: '(' action_embed_word ')' final { $$->action = $2->action; }; +action_embed: action_embed_block final { $$->action = $1->action; }; + +nonterm action_embed_word uses action_ref; + +action_embed_word: + TK_Word final { + /* Set the name in the actionDict. */ + Action *action = pd->actionDict.find( $1->data ); + if ( action != 0 ) { + /* Pass up the action element */ + $$->action = action; + } + else { + /* Will recover by returning null as the action. */ + error($1->loc) << "action lookup of \"" << $1->data << "\" failed" << endl; + $$->action = 0; + } + }; + +nonterm action_embed_block uses action_ref; + +action_embed_block: + '{' inline_block '}' final { + /* Create the action, add it to the list and pass up. */ + Action *newAction = new Action( $1->loc, 0, $2->inlineList, pd->nextCondId++ ); + pd->actionList.append( newAction ); + $$->action = newAction; + }; + +nonterm priority_name +{ + int priorityName; +}; + +# A specified priority name. Looks up the name in the current priority +# dictionary. +priority_name: + TK_Word final { + // Lookup/create the priority key. + PriorDictEl *priorDictEl; + if ( pd->priorDict.insert( $1->data, pd->nextPriorKey, &priorDictEl ) ) + pd->nextPriorKey += 1; + + // Use the inserted/found priority key. + $$->priorityName = priorDictEl->value; + }; + +nonterm priority_aug +{ + int priorityNum; +}; + +# Priority change specs. +priority_aug: + priority_aug_num final { + // Convert the priority number to a long. Check for overflow. + errno = 0; + //cerr << "PRIOR AUG: " << $1->token.data << endl; + long aug = strtol( $1->token.data, 0, 10 ); + if ( errno == ERANGE && aug == LONG_MAX ) { + /* Priority number too large. Recover by setting the priority to 0. */ + error($1->token.loc) << "priority number " << $1->token.data << + " overflows" << endl; + $$->priorityNum = 0; + } + else if ( errno == ERANGE && aug == LONG_MIN ) { + /* Priority number too large in the neg. Recover by using 0. */ + error($1->token.loc) << "priority number " << $1->token.data << + " underflows" << endl; + $$->priorityNum = 0; + } + else { + /* No overflow or underflow. */ + $$->priorityNum = aug; + } + }; + +nonterm priority_aug_num uses token_type; + +priority_aug_num: + TK_UInt final { + $$->token = *$1; + }; +priority_aug_num: + '+' TK_UInt final { + $$->token.set( "+", 1 ); + $$->token.loc = $1->loc; + $$->token.append( *$2 ); + }; +priority_aug_num: + '-' TK_UInt final { + $$->token.set( "-", 1 ); + $$->token.loc = $1->loc; + $$->token.append( *$2 ); + }; + +nonterm local_err_name +{ + int error_name; +}; + +local_err_name: + TK_Word final { + /* Lookup/create the priority key. */ + LocalErrDictEl *localErrDictEl; + if ( pd->localErrDict.insert( $1->data, pd->nextLocalErrKey, &localErrDictEl ) ) + pd->nextLocalErrKey += 1; + + /* Use the inserted/found priority key. */ + $$->error_name = localErrDictEl->value; + }; + + + +# The fourth level of precedence. These are the trailing unary operators that +# allow for repetition. + +nonterm factor_with_rep +{ + FactorWithRep *factorWithRep; +}; + +factor_with_rep: + factor_with_rep '*' final { + $$->factorWithRep = new FactorWithRep( $2->loc, $1->factorWithRep, + 0, 0, FactorWithRep::StarType ); + }; +factor_with_rep: + factor_with_rep TK_StarStar final { + $$->factorWithRep = new FactorWithRep( $2->loc, $1->factorWithRep, + 0, 0, FactorWithRep::StarStarType ); + }; +factor_with_rep: + factor_with_rep '?' final { + $$->factorWithRep = new FactorWithRep( $2->loc, $1->factorWithRep, + 0, 0, FactorWithRep::OptionalType ); + }; +factor_with_rep: + factor_with_rep '+' final { + $$->factorWithRep = new FactorWithRep( $2->loc, $1->factorWithRep, + 0, 0, FactorWithRep::PlusType ); + }; +factor_with_rep: + factor_with_rep '{' factor_rep_num '}' final { + $$->factorWithRep = new FactorWithRep( $2->loc, $1->factorWithRep, + $3->rep, 0, FactorWithRep::ExactType ); + }; +factor_with_rep: + factor_with_rep '{' ',' factor_rep_num '}' final { + $$->factorWithRep = new FactorWithRep( $2->loc, $1->factorWithRep, + 0, $4->rep, FactorWithRep::MaxType ); + }; +factor_with_rep: + factor_with_rep '{' factor_rep_num ',' '}' final { + $$->factorWithRep = new FactorWithRep( $2->loc, $1->factorWithRep, + $3->rep, 0, FactorWithRep::MinType ); + }; +factor_with_rep: + factor_with_rep '{' factor_rep_num ',' factor_rep_num '}' final { + $$->factorWithRep = new FactorWithRep( $2->loc, $1->factorWithRep, + $3->rep, $5->rep, FactorWithRep::RangeType ); + }; +factor_with_rep: + factor_with_neg final { + $$->factorWithRep = new FactorWithRep( $1->factorWithNeg ); + }; + +nonterm factor_rep_num +{ + int rep; +}; + +factor_rep_num: + TK_UInt final { + // Convert the priority number to a long. Check for overflow. + errno = 0; + long rep = strtol( $1->data, 0, 10 ); + if ( errno == ERANGE && rep == LONG_MAX ) { + // Repetition too large. Recover by returing repetition 1. */ + error($1->loc) << "repetition number " << $1->data << " overflows" << endl; + $$->rep = 1; + } + else { + // Cannot be negative, so no overflow. + $$->rep = rep; + } + }; + + +# +# The fifth level up in precedence. Negation. +# + +nonterm factor_with_neg +{ + FactorWithNeg *factorWithNeg; +}; + +factor_with_neg: + '!' factor_with_neg final { + $$->factorWithNeg = new FactorWithNeg( $1->loc, + $2->factorWithNeg, FactorWithNeg::NegateType ); + }; +factor_with_neg: + '^' factor_with_neg final { + $$->factorWithNeg = new FactorWithNeg( $1->loc, + $2->factorWithNeg, FactorWithNeg::CharNegateType ); + }; +factor_with_neg: + factor final { + $$->factorWithNeg = new FactorWithNeg( $1->factor ); + }; + +nonterm factor +{ + Factor *factor; +}; + +factor: + TK_Literal final { + /* Create a new factor node going to a concat literal. */ + $$->factor = new Factor( new Literal( *$1, Literal::LitString ) ); + }; +factor: + alphabet_num final { + /* Create a new factor node going to a literal number. */ + $$->factor = new Factor( new Literal( $1->token, Literal::Number ) ); + }; +factor: + TK_Word final { + /* Find the named graph. */ + GraphDictEl *gdNode = pd->graphDict.find( $1->data ); + if ( gdNode == 0 ) { + /* Recover by returning null as the factor node. */ + error($1->loc) << "graph lookup of \"" << $1->data << "\" failed" << endl; + $$->factor = 0; + } + else if ( gdNode->isInstance ) { + /* Recover by retuning null as the factor node. */ + error($1->loc) << "references to graph instantiations not allowed " + "in expressions" << endl; + $$->factor = 0; + } + else { + /* Create a factor node that is a lookup of an expression. */ + $$->factor = new Factor( $1->loc, gdNode->value ); + } + }; +factor: + RE_SqOpen regular_expr_or_data RE_SqClose final { + /* Create a new factor node going to an OR expression. */ + $$->factor = new Factor( new ReItem( $1->loc, $2->reOrBlock, ReItem::OrBlock ) ); + }; +factor: + RE_SqOpenNeg regular_expr_or_data RE_SqClose final { + /* Create a new factor node going to a negated OR expression. */ + $$->factor = new Factor( new ReItem( $1->loc, $2->reOrBlock, ReItem::NegOrBlock ) ); + }; +factor: + RE_Slash regular_expr RE_Slash final { + if ( $3->length > 1 ) { + for ( char *p = $3->data; *p != 0; p++ ) { + if ( *p == 'i' ) + $2->regExpr->caseInsensitive = true; + } + } + + /* Create a new factor node going to a regular exp. */ + $$->factor = new Factor( $2->regExpr ); + }; +factor: + range_lit TK_DotDot range_lit final { + /* Create a new factor node going to a range. */ + $$->factor = new Factor( new Range( $1->literal, $3->literal ) ); + }; +factor: + '(' join ')' final { + /* Create a new factor going to a parenthesized join. */ + $$->factor = new Factor( $2->join ); + $2->join->loc = $1->loc; + }; + +nonterm range_lit +{ + Literal *literal; +}; + +# Literals which can be the end points of ranges. +range_lit: + TK_Literal final { + /* Range literas must have only one char. We restrict this in the parse tree. */ + $$->literal = new Literal( *$1, Literal::LitString ); + }; +range_lit: + alphabet_num final { + /* Create a new literal number. */ + $$->literal = new Literal( $1->token, Literal::Number ); + }; + +nonterm alphabet_num uses token_type; + +# Any form of a number that can be used as a basic machine. */ +alphabet_num: + TK_UInt final { + $$->token = *$1; + }; +alphabet_num: + '-' TK_UInt final { + $$->token.set( "-", 1 ); + $$->token.loc = $1->loc; + $$->token.append( *$2 ); + }; +alphabet_num: + TK_Hex final { + $$->token = *$1; + }; +# +# Regular Expressions. +# + +nonterm regular_expr +{ + RegExpr *regExpr; +}; + +# Parser for regular expression fsms. Any number of expression items which +# generally gives a machine one character long or one character long stared. +regular_expr: + regular_expr regular_expr_item final { + /* An optimization to lessen the tree size. If a non-starred char is + * directly under the left side on the right and the right side is + * another non-starred char then paste them together and return the + * left side. Otherwise just put the two under a new reg exp node. */ + if ( $2->reItem->type == ReItem::Data && !$2->reItem->star && + $1->regExpr->type == RegExpr::RecurseItem && + $1->regExpr->item->type == ReItem::Data && !$1->regExpr->item->star ) + { + /* Append the right side to the right side of the left and toss the + * right side. */ + $1->regExpr->item->token.append( $2->reItem->token ); + delete $2->reItem; + $$->regExpr = $1->regExpr; + } + else { + $$->regExpr = new RegExpr( $1->regExpr, $2->reItem ); + } + }; +regular_expr: + final { + /* Can't optimize the tree. */ + $$->regExpr = new RegExpr(); + }; + +nonterm regular_expr_item +{ + ReItem *reItem; +}; + +# RegularExprItems can be a character spec with an optional staring of the char. +regular_expr_item: + regular_expr_char RE_Star final { + $1->reItem->star = true; + $$->reItem = $1->reItem; + }; +regular_expr_item: + regular_expr_char final { + $$->reItem = $1->reItem; + }; + +nonterm regular_expr_char +{ + ReItem *reItem; +}; + +# A character spec can be a set of characters inside of square parenthesis, a +# dot specifying any character or some explicitly stated character. +regular_expr_char: + RE_SqOpen regular_expr_or_data RE_SqClose final { + $$->reItem = new ReItem( $1->loc, $2->reOrBlock, ReItem::OrBlock ); + }; +regular_expr_char: + RE_SqOpenNeg regular_expr_or_data RE_SqClose final { + $$->reItem = new ReItem( $1->loc, $2->reOrBlock, ReItem::NegOrBlock ); + }; +regular_expr_char: + RE_Dot final { + $$->reItem = new ReItem( $1->loc, ReItem::Dot ); + }; +regular_expr_char: + RE_Char final { + $$->reItem = new ReItem( $1->loc, *$1 ); + }; + +# The data inside of a [] expression in a regular expression. Accepts any +# number of characters or ranges. */ +nonterm regular_expr_or_data +{ + ReOrBlock *reOrBlock; +}; + +regular_expr_or_data: + regular_expr_or_data regular_expr_or_char final { + /* An optimization to lessen the tree size. If an or char is directly + * under the left side on the right and the right side is another or + * char then paste them together and return the left side. Otherwise + * just put the two under a new or data node. */ + if ( $2->reOrItem->type == ReOrItem::Data && + $1->reOrBlock->type == ReOrBlock::RecurseItem && + $1->reOrBlock->item->type == ReOrItem::Data ) + { + /* Append the right side to right side of the left and toss the + * right side. */ + $1->reOrBlock->item->token.append( $2->reOrItem->token ); + delete $2->reOrItem; + $$->reOrBlock = $1->reOrBlock; + } + else { + /* Can't optimize, put the left and right under a new node. */ + $$->reOrBlock = new ReOrBlock( $1->reOrBlock, $2->reOrItem ); + } + }; +regular_expr_or_data: + final { + $$->reOrBlock = new ReOrBlock(); + }; + +# A single character inside of an or expression. Can either be a character or a +# set of characters. +nonterm regular_expr_or_char +{ + ReOrItem *reOrItem; +}; + +regular_expr_or_char: + RE_Char final { + $$->reOrItem = new ReOrItem( $1->loc, *$1 ); + }; +regular_expr_or_char: + RE_Char RE_Dash RE_Char final { + $$->reOrItem = new ReOrItem( $2->loc, $1->data[0], $3->data[0] ); + }; + +# +# Inline Lists for inline host code. +# + +type inline_list +{ + InlineList *inlineList; +}; + +nonterm inline_block uses inline_list; + +inline_block: + inline_block inline_block_item + final { + /* Append the item to the list, return the list. */ + $$->inlineList = $1->inlineList; + $$->inlineList->append( $2->inlineItem ); + }; + +inline_block: + final { + /* Start with empty list. */ + $$->inlineList = new InlineList; + }; + +type inline_item +{ + InlineItem *inlineItem; +}; + +nonterm inline_block_item uses inline_item; +nonterm inline_block_interpret uses inline_item; + +inline_block_item: + inline_expr_any + final { + $$->inlineItem = new InlineItem( $1->token.loc, $1->token.data, InlineItem::Text ); + }; + +inline_block_item: + inline_block_symbol + final { + $$->inlineItem = new InlineItem( $1->token.loc, $1->token.data, InlineItem::Text ); + }; + +inline_block_item: + inline_block_interpret + final { + /* Pass the inline item up. */ + $$->inlineItem = $1->inlineItem; + }; + +nonterm inline_block_symbol uses token_type; + +inline_block_symbol: ',' final { $$->token = *$1; }; +inline_block_symbol: ';' final { $$->token = *$1; }; +inline_block_symbol: '(' final { $$->token = *$1; }; +inline_block_symbol: ')' final { $$->token = *$1; }; +inline_block_symbol: '*' final { $$->token = *$1; }; +inline_block_symbol: TK_NameSep final { $$->token = *$1; }; + +# Interpreted statements in a struct block. */ +inline_block_interpret: + inline_expr_interpret final { + /* Pass up interpreted items of inline expressions. */ + $$->inlineItem = $1->inlineItem; + }; +inline_block_interpret: + KW_Hold ';' final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::Hold ); + }; +inline_block_interpret: + KW_Exec inline_expr ';' final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::Exec ); + $$->inlineItem->children = $2->inlineList; + }; +inline_block_interpret: + KW_Goto state_ref ';' final { + $$->inlineItem = new InlineItem( $1->loc, + new NameRef(nameRef), InlineItem::Goto ); + }; +inline_block_interpret: + KW_Goto '*' inline_expr ';' final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::GotoExpr ); + $$->inlineItem->children = $3->inlineList; + }; +inline_block_interpret: + KW_Next state_ref ';' final { + $$->inlineItem = new InlineItem( $1->loc, new NameRef(nameRef), InlineItem::Next ); + }; +inline_block_interpret: + KW_Next '*' inline_expr ';' final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::NextExpr ); + $$->inlineItem->children = $3->inlineList; + }; +inline_block_interpret: + KW_Call state_ref ';' final { + $$->inlineItem = new InlineItem( $1->loc, new NameRef(nameRef), InlineItem::Call ); + }; +inline_block_interpret: + KW_Call '*' inline_expr ';' final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::CallExpr ); + $$->inlineItem->children = $3->inlineList; + }; +inline_block_interpret: + KW_Ret ';' final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::Ret ); + }; +inline_block_interpret: + KW_Break ';' final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::Break ); + }; + +nonterm inline_expr uses inline_list; + +inline_expr: + inline_expr inline_expr_item + final { + $$->inlineList = $1->inlineList; + $$->inlineList->append( $2->inlineItem ); + }; +inline_expr: + final { + /* Init the list used for this expr. */ + $$->inlineList = new InlineList; + }; + +nonterm inline_expr_item uses inline_item; + +inline_expr_item: + inline_expr_any + final { + /* Return a text segment. */ + $$->inlineItem = new InlineItem( $1->token.loc, $1->token.data, InlineItem::Text ); + }; +inline_expr_item: + inline_expr_symbol + final { + /* Return a text segment, must heap alloc the text. */ + $$->inlineItem = new InlineItem( $1->token.loc, $1->token.data, InlineItem::Text ); + }; +inline_expr_item: + inline_expr_interpret + final{ + /* Pass the inline item up. */ + $$->inlineItem = $1->inlineItem; + }; + +nonterm inline_expr_any uses token_type; + +inline_expr_any: IL_WhiteSpace try { $$->token = *$1; }; +inline_expr_any: IL_Comment try { $$->token = *$1; }; +inline_expr_any: IL_Literal try { $$->token = *$1; }; +inline_expr_any: IL_Symbol try { $$->token = *$1; }; +inline_expr_any: TK_UInt try { $$->token = *$1; }; +inline_expr_any: TK_Hex try { $$->token = *$1; }; +inline_expr_any: TK_Word try { $$->token = *$1; }; + +# Anything in a ExecValExpr that is not dynamically allocated. This includes +# all special symbols caught in inline code except the semi. + +nonterm inline_expr_symbol uses token_type; + +inline_expr_symbol: ',' try { $$->token = *$1; }; +inline_expr_symbol: '(' try { $$->token = *$1; }; +inline_expr_symbol: ')' try { $$->token = *$1; }; +inline_expr_symbol: '*' try { $$->token = *$1; }; +inline_expr_symbol: TK_NameSep try { $$->token = *$1; }; + +nonterm inline_expr_interpret uses inline_item; + +inline_expr_interpret: + KW_PChar + final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::PChar ); + }; +inline_expr_interpret: + KW_Char + final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::Char ); + }; +inline_expr_interpret: + KW_CurState + final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::Curs ); + }; +inline_expr_interpret: + KW_TargState + final { + $$->inlineItem = new InlineItem( $1->loc, InlineItem::Targs ); + }; +inline_expr_interpret: + KW_Entry '(' state_ref ')' + final { + $$->inlineItem = new InlineItem( $1->loc, + new NameRef(nameRef), InlineItem::Entry ); + }; + +# A local state reference. Cannot have :: prefix. +local_state_ref: + no_name_sep state_ref_names; + +# Clear the name ref structure. +no_name_sep: + final { + nameRef.empty(); + }; + +# A qualified state reference. +state_ref: opt_name_sep state_ref_names; + +# Optional leading name separator. +opt_name_sep: + TK_NameSep + final { + /* Insert an initial null pointer val to indicate the existence of the + * initial name seperator. */ + nameRef.setAs( 0 ); + }; +opt_name_sep: + final { + nameRef.empty(); + }; + +# List of names separated by :: +state_ref_names: + state_ref_names TK_NameSep TK_Word + final { + nameRef.append( $3->data ); + }; +state_ref_names: + TK_Word + final { + nameRef.append( $1->data ); + }; + +}%% + +%%{ + write types; + write data; +}%% + +void Parser::init() +{ + %% write init; +} + +int Parser::parseLangEl( int type, const Token *token ) +{ + %% write exec; + return errCount == 0 ? 0 : -1; +} + +void Parser::tryMachineDef( InputLoc &loc, char *name, + MachineDef *machineDef, bool isInstance ) +{ + GraphDictEl *newEl = pd->graphDict.insert( name ); + if ( newEl != 0 ) { + /* New element in the dict, all good. */ + newEl->value = new VarDef( name, machineDef ); + newEl->isInstance = isInstance; + newEl->loc = loc; + newEl->value->isExport = exportContext[exportContext.length()-1]; + + /* It it is an instance, put on the instance list. */ + if ( isInstance ) + pd->instanceList.append( newEl ); + } + else { + // Recover by ignoring the duplicate. + error(loc) << "fsm \"" << name << "\" previously defined" << endl; + } +} + +ostream &Parser::parse_error( int tokId, Token &token ) +{ + /* Maintain the error count. */ + gblErrorCount += 1; + + cerr << token.loc << ": "; + cerr << "at token "; + if ( tokId < 128 ) + cerr << "\"" << Parser_lelNames[tokId] << "\""; + else + cerr << Parser_lelNames[tokId]; + if ( token.data != 0 ) + cerr << " with data \"" << token.data << "\""; + cerr << ": "; + + return cerr; +} + +int Parser::token( InputLoc &loc, int tokId, char *tokstart, int toklen ) +{ + Token token; + token.data = tokstart; + token.length = toklen; + token.loc = loc; + int res = parseLangEl( tokId, &token ); + if ( res < 0 ) { + parse_error(tokId, token) << "parse error" << endl; + exit(1); + } + return res; +} diff --git a/contrib/tools/ragel6/rlscan.cpp b/contrib/tools/ragel6/rlscan.cpp new file mode 100644 index 0000000000..5e2fc36645 --- /dev/null +++ b/contrib/tools/ragel6/rlscan.cpp @@ -0,0 +1,7231 @@ + +#line 1 "rlscan.rl" +/* + * Copyright 2006-2007 Adrian Thurston <thurston@complang.org> + * Copyright 2011 Josef Goettgens + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iostream> +#include <fstream> +#include <string.h> + +#include "ragel.h" +#include "rlscan.h" +#include "inputdata.h" + +//#define LOG_TOKENS + +using std::ifstream; +using std::istream; +using std::ostream; +using std::cout; +using std::cerr; +using std::endl; + +enum InlineBlockType +{ + CurlyDelimited, + SemiTerminated +}; + + +/* + * The Scanner for Importing + */ + + +#line 125 "rlscan.rl" + + + +#line 65 "rlscan.cpp" +static const int inline_token_scan_start = 2; +static const int inline_token_scan_first_final = 2; +static const int inline_token_scan_error = -1; + +static const int inline_token_scan_en_main = 2; + + +#line 128 "rlscan.rl" + +void Scanner::flushImport() +{ + int *p = token_data; + int *pe = token_data + cur_token; + int *eof = 0; + + +#line 82 "rlscan.cpp" + { + tok_cs = inline_token_scan_start; + tok_ts = 0; + tok_te = 0; + tok_act = 0; + } + +#line 90 "rlscan.cpp" + { + if ( p == pe ) + goto _test_eof; + switch ( tok_cs ) + { +tr0: +#line 123 "rlscan.rl" + {{p = (( tok_te))-1;}} + goto st2; +tr1: +#line 109 "rlscan.rl" + { tok_te = p+1;{ + int base = tok_ts - token_data; + int nameOff = 0; + int litOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_Literal, + token_strings[base+litOff], token_lens[base+litOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }} + goto st2; +tr2: +#line 81 "rlscan.rl" + { tok_te = p+1;{ + int base = tok_ts - token_data; + int nameOff = 0; + int numOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_UInt, + token_strings[base+numOff], token_lens[base+numOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }} + goto st2; +tr3: +#line 95 "rlscan.rl" + { tok_te = p+1;{ + int base = tok_ts - token_data; + int nameOff = 1; + int litOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_Literal, + token_strings[base+litOff], token_lens[base+litOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }} + goto st2; +tr4: +#line 67 "rlscan.rl" + { tok_te = p+1;{ + int base = tok_ts - token_data; + int nameOff = 1; + int numOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_UInt, + token_strings[base+numOff], token_lens[base+numOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }} + goto st2; +tr5: +#line 123 "rlscan.rl" + { tok_te = p+1;} + goto st2; +tr8: +#line 123 "rlscan.rl" + { tok_te = p;p--;} + goto st2; +st2: +#line 1 "NONE" + { tok_ts = 0;} + if ( ++p == pe ) + goto _test_eof2; +case 2: +#line 1 "NONE" + { tok_ts = p;} +#line 176 "rlscan.cpp" + switch( (*p) ) { + case 128: goto tr6; + case 131: goto tr7; + } + goto tr5; +tr6: +#line 1 "NONE" + { tok_te = p+1;} + goto st3; +st3: + if ( ++p == pe ) + goto _test_eof3; +case 3: +#line 190 "rlscan.cpp" + if ( (*p) == 61 ) + goto st0; + goto tr8; +st0: + if ( ++p == pe ) + goto _test_eof0; +case 0: + switch( (*p) ) { + case 129: goto tr1; + case 130: goto tr2; + } + goto tr0; +tr7: +#line 1 "NONE" + { tok_te = p+1;} + goto st4; +st4: + if ( ++p == pe ) + goto _test_eof4; +case 4: +#line 211 "rlscan.cpp" + if ( (*p) == 128 ) + goto st1; + goto tr8; +st1: + if ( ++p == pe ) + goto _test_eof1; +case 1: + switch( (*p) ) { + case 129: goto tr3; + case 130: goto tr4; + } + goto tr0; + } + _test_eof2: tok_cs = 2; goto _test_eof; + _test_eof3: tok_cs = 3; goto _test_eof; + _test_eof0: tok_cs = 0; goto _test_eof; + _test_eof4: tok_cs = 4; goto _test_eof; + _test_eof1: tok_cs = 1; goto _test_eof; + + _test_eof: {} + if ( p == eof ) + { + switch ( tok_cs ) { + case 3: goto tr8; + case 0: goto tr0; + case 4: goto tr8; + case 1: goto tr0; + } + } + + } + +#line 139 "rlscan.rl" + + + if ( tok_ts == 0 ) + cur_token = 0; + else { + cur_token = pe - tok_ts; + int ts_offset = tok_ts - token_data; + memmove( token_data, token_data+ts_offset, cur_token*sizeof(token_data[0]) ); + memmove( token_strings, token_strings+ts_offset, cur_token*sizeof(token_strings[0]) ); + memmove( token_lens, token_lens+ts_offset, cur_token*sizeof(token_lens[0]) ); + } +} + +void Scanner::directToParser( Parser *toParser, const char *tokFileName, int tokLine, + int tokColumn, int type, char *tokdata, int toklen ) +{ + InputLoc loc; + + #ifdef LOG_TOKENS + cerr << "scanner:" << tokLine << ":" << tokColumn << + ": sending token to the parser " << Parser_lelNames[type]; + cerr << " " << toklen; + if ( tokdata != 0 ) + cerr << " " << tokdata; + cerr << endl; + #endif + + loc.fileName = tokFileName; + loc.line = tokLine; + loc.col = tokColumn; + + toParser->token( loc, type, tokdata, toklen ); +} + +void Scanner::importToken( int token, char *start, char *end ) +{ + if ( cur_token == max_tokens ) + flushImport(); + + token_data[cur_token] = token; + if ( start == 0 ) { + token_strings[cur_token] = 0; + token_lens[cur_token] = 0; + } + else { + int toklen = end-start; + token_lens[cur_token] = toklen; + token_strings[cur_token] = new char[toklen+1]; + memcpy( token_strings[cur_token], start, toklen ); + token_strings[cur_token][toklen] = 0; + } + cur_token++; +} + +void Scanner::pass( int token, char *start, char *end ) +{ + if ( importMachines ) + importToken( token, start, end ); + pass(); +} + +void Scanner::pass() +{ + updateCol(); + + /* If no errors and we are at the bottom of the include stack (the + * source file listed on the command line) then write out the data. */ + if ( includeDepth == 0 && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->data.write( ts, te-ts ); +} + +/* + * The scanner for processing sections, includes, imports, etc. + */ + + +#line 321 "rlscan.cpp" +static const int section_parse_start = 10; +static const int section_parse_first_final = 10; +static const int section_parse_error = 0; + +static const int section_parse_en_main = 10; + + +#line 218 "rlscan.rl" + + + +void Scanner::init( ) +{ + +#line 336 "rlscan.cpp" + { + cs = section_parse_start; + } + +#line 224 "rlscan.rl" +} + +bool Scanner::active() +{ + if ( ignoreSection ) + return false; + + if ( parser == 0 && ! parserExistsError ) { + scan_error() << "this specification has no name, nor does any previous" + " specification" << endl; + parserExistsError = true; + } + + if ( parser == 0 ) + return false; + + return true; +} + +ostream &Scanner::scan_error() +{ + /* Maintain the error count. */ + gblErrorCount += 1; + cerr << makeInputLoc( fileName, line, column ) << ": "; + return cerr; +} + +/* An approximate check for duplicate includes. Due to aliasing of files it's + * possible for duplicates to creep in. */ +bool Scanner::duplicateInclude( char *inclFileName, char *inclSectionName ) +{ + for ( IncludeHistory::Iter hi = parser->includeHistory; hi.lte(); hi++ ) { + if ( strcmp( hi->fileName, inclFileName ) == 0 && + strcmp( hi->sectionName, inclSectionName ) == 0 ) + { + return true; + } + } + return false; +} + +void Scanner::updateCol() +{ + char *from = lastnl; + if ( from == 0 ) + from = ts; + //cerr << "adding " << te - from << " to column" << endl; + column += te - from; + lastnl = 0; +} + +void Scanner::handleMachine() +{ + /* Assign a name to the machine. */ + char *machine = word; + + if ( !importMachines && inclSectionTarg == 0 ) { + ignoreSection = false; + + ParserDictEl *pdEl = id.parserDict.find( machine ); + if ( pdEl == 0 ) { + pdEl = new ParserDictEl( machine ); + pdEl->value = new Parser( fileName, machine, sectionLoc ); + pdEl->value->init(); + id.parserDict.insert( pdEl ); + id.parserList.append( pdEl->value ); + } + + parser = pdEl->value; + } + else if ( !importMachines && strcmp( inclSectionTarg, machine ) == 0 ) { + /* found include target */ + ignoreSection = false; + parser = inclToParser; + } + else { + /* ignoring section */ + ignoreSection = true; + parser = 0; + } +} + +void Scanner::handleInclude() +{ + if ( active() ) { + char *inclSectionName = word; + char **includeChecks = 0; + + /* Implement defaults for the input file and section name. */ + if ( inclSectionName == 0 ) + inclSectionName = parser->sectionName; + + if ( lit != 0 ) + includeChecks = makeIncludePathChecks( fileName, lit, lit_len ); + else { + char *test = new char[strlen(fileName)+1]; + strcpy( test, fileName ); + + includeChecks = new char*[2]; + + includeChecks[0] = test; + includeChecks[1] = 0; + } + + long found = 0; + ifstream *inFile = tryOpenInclude( includeChecks, found ); + if ( inFile == 0 ) { + scan_error() << "include: failed to locate file" << endl; + char **tried = includeChecks; + while ( *tried != 0 ) + scan_error() << "include: attempted: \"" << *tried++ << '\"' << endl; + } + else { + /* Don't include anything that's already been included. */ + if ( !duplicateInclude( includeChecks[found], inclSectionName ) ) { + parser->includeHistory.append( IncludeHistoryItem( + includeChecks[found], inclSectionName ) ); + + Scanner scanner( id, includeChecks[found], *inFile, parser, + inclSectionName, includeDepth+1, false ); + scanner.do_scan( ); + delete inFile; + } + } + } +} + +void Scanner::handleImport() +{ + if ( active() ) { + char **importChecks = makeIncludePathChecks( fileName, lit, lit_len ); + + /* Open the input file for reading. */ + long found = 0; + ifstream *inFile = tryOpenInclude( importChecks, found ); + if ( inFile == 0 ) { + scan_error() << "import: could not open import file " << + "for reading" << endl; + char **tried = importChecks; + while ( *tried != 0 ) + scan_error() << "import: attempted: \"" << *tried++ << '\"' << endl; + } + + Scanner scanner( id, importChecks[found], *inFile, parser, + 0, includeDepth+1, true ); + scanner.do_scan( ); + scanner.importToken( 0, 0, 0 ); + scanner.flushImport(); + delete inFile; + } +} + + +#line 461 "rlscan.rl" + + +void Scanner::token( int type, char c ) +{ + token( type, &c, &c + 1 ); +} + +void Scanner::token( int type ) +{ + token( type, 0, 0 ); +} + +void Scanner::token( int type, char *start, char *end ) +{ + char *tokdata = 0; + int toklen = 0; + if ( start != 0 ) { + toklen = end-start; + tokdata = new char[toklen+1]; + memcpy( tokdata, start, toklen ); + tokdata[toklen] = 0; + } + + processToken( type, tokdata, toklen ); +} + +void Scanner::processToken( int type, char *tokdata, int toklen ) +{ + int *p, *pe, *eof; + + if ( type < 0 ) + p = pe = eof = 0; + else { + p = &type; + pe = &type + 1; + eof = 0; + } + + +#line 535 "rlscan.cpp" + { + if ( p == pe ) + goto _test_eof; + switch ( cs ) + { +tr2: +#line 391 "rlscan.rl" + { handleMachine(); } + goto st10; +tr6: +#line 392 "rlscan.rl" + { handleInclude(); } + goto st10; +tr10: +#line 393 "rlscan.rl" + { handleImport(); } + goto st10; +tr13: +#line 433 "rlscan.rl" + { + if ( active() && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->writeArgs.append( 0 ); + } + goto st10; +tr14: +#line 444 "rlscan.rl" + { + /* Send the token off to the parser. */ + if ( active() ) + directToParser( parser, fileName, line, column, type, tokdata, toklen ); + } + goto st10; +st10: + if ( ++p == pe ) + goto _test_eof10; +case 10: +#line 572 "rlscan.cpp" + switch( (*p) ) { + case 191: goto st1; + case 192: goto st3; + case 193: goto st6; + case 194: goto tr18; + } + goto tr14; +st1: + if ( ++p == pe ) + goto _test_eof1; +case 1: + if ( (*p) == 128 ) + goto tr1; + goto tr0; +tr0: +#line 386 "rlscan.rl" + { scan_error() << "bad machine statement" << endl; } + goto st0; +tr3: +#line 387 "rlscan.rl" + { scan_error() << "bad include statement" << endl; } + goto st0; +tr8: +#line 388 "rlscan.rl" + { scan_error() << "bad import statement" << endl; } + goto st0; +tr11: +#line 389 "rlscan.rl" + { scan_error() << "bad write statement" << endl; } + goto st0; +#line 603 "rlscan.cpp" +st0: +cs = 0; + goto _out; +tr1: +#line 383 "rlscan.rl" + { word = tokdata; word_len = toklen; } + goto st2; +st2: + if ( ++p == pe ) + goto _test_eof2; +case 2: +#line 615 "rlscan.cpp" + if ( (*p) == 59 ) + goto tr2; + goto tr0; +st3: + if ( ++p == pe ) + goto _test_eof3; +case 3: + switch( (*p) ) { + case 128: goto tr4; + case 129: goto tr5; + } + goto tr3; +tr4: +#line 382 "rlscan.rl" + { word = lit = 0; word_len = lit_len = 0; } +#line 383 "rlscan.rl" + { word = tokdata; word_len = toklen; } + goto st4; +st4: + if ( ++p == pe ) + goto _test_eof4; +case 4: +#line 638 "rlscan.cpp" + switch( (*p) ) { + case 59: goto tr6; + case 129: goto tr7; + } + goto tr3; +tr5: +#line 382 "rlscan.rl" + { word = lit = 0; word_len = lit_len = 0; } +#line 384 "rlscan.rl" + { lit = tokdata; lit_len = toklen; } + goto st5; +tr7: +#line 384 "rlscan.rl" + { lit = tokdata; lit_len = toklen; } + goto st5; +st5: + if ( ++p == pe ) + goto _test_eof5; +case 5: +#line 658 "rlscan.cpp" + if ( (*p) == 59 ) + goto tr6; + goto tr3; +st6: + if ( ++p == pe ) + goto _test_eof6; +case 6: + if ( (*p) == 129 ) + goto tr9; + goto tr8; +tr9: +#line 384 "rlscan.rl" + { lit = tokdata; lit_len = toklen; } + goto st7; +st7: + if ( ++p == pe ) + goto _test_eof7; +case 7: +#line 677 "rlscan.cpp" + if ( (*p) == 59 ) + goto tr10; + goto tr8; +tr18: +#line 413 "rlscan.rl" + { + if ( active() && machineSpec == 0 && machineName == 0 ) { + InputItem *inputItem = new InputItem; + inputItem->type = InputItem::Write; + inputItem->loc.fileName = fileName; + inputItem->loc.line = line; + inputItem->loc.col = column; + inputItem->name = parser->sectionName; + inputItem->pd = parser->pd; + id.inputItems.append( inputItem ); + } + } + goto st8; +st8: + if ( ++p == pe ) + goto _test_eof8; +case 8: +#line 700 "rlscan.cpp" + if ( (*p) == 128 ) + goto tr12; + goto tr11; +tr12: +#line 427 "rlscan.rl" + { + if ( active() && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->writeArgs.append( strdup(tokdata) ); + } + goto st9; +st9: + if ( ++p == pe ) + goto _test_eof9; +case 9: +#line 715 "rlscan.cpp" + switch( (*p) ) { + case 59: goto tr13; + case 128: goto tr12; + } + goto tr11; + } + _test_eof10: cs = 10; goto _test_eof; + _test_eof1: cs = 1; goto _test_eof; + _test_eof2: cs = 2; goto _test_eof; + _test_eof3: cs = 3; goto _test_eof; + _test_eof4: cs = 4; goto _test_eof; + _test_eof5: cs = 5; goto _test_eof; + _test_eof6: cs = 6; goto _test_eof; + _test_eof7: cs = 7; goto _test_eof; + _test_eof8: cs = 8; goto _test_eof; + _test_eof9: cs = 9; goto _test_eof; + + _test_eof: {} + if ( p == eof ) + { + switch ( cs ) { + case 1: + case 2: +#line 386 "rlscan.rl" + { scan_error() << "bad machine statement" << endl; } + break; + case 3: + case 4: + case 5: +#line 387 "rlscan.rl" + { scan_error() << "bad include statement" << endl; } + break; + case 6: + case 7: +#line 388 "rlscan.rl" + { scan_error() << "bad import statement" << endl; } + break; + case 8: + case 9: +#line 389 "rlscan.rl" + { scan_error() << "bad write statement" << endl; } + break; +#line 758 "rlscan.cpp" + } + } + + _out: {} + } + +#line 502 "rlscan.rl" + + + updateCol(); + + /* Record the last token for use in controlling the scan of subsequent + * tokens. */ + lastToken = type; +} + +void Scanner::startSection( ) +{ + parserExistsError = false; + + sectionLoc.fileName = fileName; + sectionLoc.line = line; + sectionLoc.col = column; +} + +void Scanner::endSection( ) +{ + /* Execute the eof actions for the section parser. */ + processToken( -1, 0, 0 ); + + /* Close off the section with the parser. */ + if ( active() ) { + InputLoc loc; + loc.fileName = fileName; + loc.line = line; + loc.col = column; + + parser->token( loc, TK_EndSection, 0, 0 ); + } + + if ( includeDepth == 0 ) { + if ( machineSpec == 0 && machineName == 0 ) { + /* The end section may include a newline on the end, so + * we use the last line, which will count the newline. */ + InputItem *inputItem = new InputItem; + inputItem->type = InputItem::HostData; + inputItem->loc.line = line; + inputItem->loc.col = column; + id.inputItems.append( inputItem ); + } + } +} + +bool isAbsolutePath( const char *path ) +{ +#ifdef _WIN32 + return isalpha( path[0] ) && path[1] == ':' && (path[2] == '\\' || path[2] == '/'); +#else + return path[0] == '/'; +#endif +} + +inline char* resolvePath(const char* rel, const char* abs) { + const size_t l1 = strlen(rel); + const size_t l2 = strlen(abs); + char* ret = new char[l1 + l2 + 1]; + + const char* p = strrchr(abs, '/') + 1; + const size_t l3 = p - abs; + + memcpy(ret, abs, l3); + strcpy(ret + l3, rel); + + return ret; +} + +char **Scanner::makeIncludePathChecks( const char *thisFileName, + const char *fileName, int fnlen ) +{ + char **checks = 0; + long nextCheck = 0; + long length = 0; + bool caseInsensitive = false; + char *data = prepareLitString( InputLoc(), fileName, fnlen, + length, caseInsensitive ); + + /* Absolute path? */ + if ( isAbsolutePath( data ) ) { + checks = new char*[2]; + checks[nextCheck++] = data; + } + else { + checks = new char*[2 + id.includePaths.length()]; + + /* Search from the the location of the current file. */ + const char *lastSlash = strrchr( thisFileName, '/' ); + if ( lastSlash == 0 ) + checks[nextCheck++] = data; + else { + checks[nextCheck++] = resolvePath(data, thisFileName); + } + + /* Search from the include paths given on the command line. */ + for ( ArgsVector::Iter incp = id.includePaths; incp.lte(); incp++ ) { + long pathLen = strlen( *incp ); + long checkLen = pathLen + 1 + length; + char *check = new char[checkLen+1]; + memcpy( check, *incp, pathLen ); + check[pathLen] = '/'; + memcpy( check+pathLen+1, data, length ); + check[checkLen] = 0; + checks[nextCheck++] = check; + } + } + + checks[nextCheck] = 0; + return checks; +} + +ifstream *Scanner::tryOpenInclude( char **pathChecks, long &found ) +{ + char **check = pathChecks; + ifstream *inFile = new ifstream; + + while ( *check != 0 ) { + inFile->open( *check ); + if ( inFile->is_open() ) { + found = check - pathChecks; + return inFile; + } + + /* + * 03/26/2011 jg: + * Don't rely on sloppy runtime behaviour: reset the state of the stream explicitly. + * If inFile->open() fails, which happens when include dirs are tested, the fail bit + * is set by the runtime library. Currently the VS runtime library opens new files, + * but when it comes to reading it refuses to work. + */ + inFile->clear(); + + check += 1; + } + + found = -1; + delete inFile; + return 0; +} + + +#line 1173 "rlscan.rl" + + + +#line 904 "rlscan.cpp" +static const int rlscan_start = 38; +static const int rlscan_first_final = 38; +static const int rlscan_error = 0; + +static const int rlscan_en_inline_code_ruby = 52; +static const int rlscan_en_inline_code = 95; +static const int rlscan_en_or_literal = 137; +static const int rlscan_en_ragel_re_literal = 139; +static const int rlscan_en_write_statement = 143; +static const int rlscan_en_parser_def = 146; +static const int rlscan_en_main_ruby = 253; +static const int rlscan_en_main = 38; + + +#line 1176 "rlscan.rl" + +void Scanner::do_scan() +{ + int bufsize = 8; + char *buf = new char[bufsize]; + int cs, act, have = 0; + int top; + + /* The stack is two deep, one level for going into ragel defs from the main + * machines which process outside code, and another for going into or literals + * from either a ragel spec, or a regular expression. */ + int stack[2]; + int curly_count = 0; + bool execute = true; + bool singleLineSpec = false; + InlineBlockType inlineBlockType = CurlyDelimited; + + /* Init the section parser and the character scanner. */ + init(); + +#line 940 "rlscan.cpp" + { + cs = rlscan_start; + top = 0; + ts = 0; + te = 0; + act = 0; + } + +#line 1196 "rlscan.rl" + + /* Set up the start state. FIXME: After 5.20 is released the nocs write + * init option should be used, the main machine eliminated and this statement moved + * above the write init. */ + if ( hostLang->lang == HostLang::Ruby ) + cs = rlscan_en_main_ruby; + else + cs = rlscan_en_main; + + while ( execute ) { + char *p = buf + have; + int space = bufsize - have; + + if ( space == 0 ) { + /* We filled up the buffer trying to scan a token. Grow it. */ + bufsize = bufsize * 2; + char *newbuf = new char[bufsize]; + + /* Recompute p and space. */ + p = newbuf + have; + space = bufsize - have; + + /* Patch up pointers possibly in use. */ + if ( ts != 0 ) + ts = newbuf + ( ts - buf ); + te = newbuf + ( te - buf ); + + /* Copy the new buffer in. */ + memcpy( newbuf, buf, have ); + delete[] buf; + buf = newbuf; + } + + input.read( p, space ); + int len = input.gcount(); + char *pe = p + len; + + /* If we see eof then append the eof var. */ + char *eof = 0; + if ( len == 0 ) { + eof = pe; + execute = false; + } + + +#line 995 "rlscan.cpp" + { + if ( p == pe ) + goto _test_eof; + goto _resume; + +_again: + switch ( cs ) { + case 38: goto st38; + case 39: goto st39; + case 40: goto st40; + case 1: goto st1; + case 2: goto st2; + case 41: goto st41; + case 42: goto st42; + case 43: goto st43; + case 3: goto st3; + case 4: goto st4; + case 44: goto st44; + case 5: goto st5; + case 6: goto st6; + case 7: goto st7; + case 45: goto st45; + case 46: goto st46; + case 47: goto st47; + case 48: goto st48; + case 49: goto st49; + case 50: goto st50; + case 51: goto st51; + case 52: goto st52; + case 53: goto st53; + case 54: goto st54; + case 8: goto st8; + case 9: goto st9; + case 55: goto st55; + case 10: goto st10; + case 56: goto st56; + case 11: goto st11; + case 12: goto st12; + case 57: goto st57; + case 13: goto st13; + case 14: goto st14; + case 58: goto st58; + case 59: goto st59; + case 15: goto st15; + case 60: goto st60; + case 61: goto st61; + case 62: goto st62; + case 63: goto st63; + case 64: goto st64; + case 65: goto st65; + case 66: goto st66; + case 67: goto st67; + case 68: goto st68; + case 69: goto st69; + case 70: goto st70; + case 71: goto st71; + case 72: goto st72; + case 73: goto st73; + case 74: goto st74; + case 75: goto st75; + case 76: goto st76; + case 77: goto st77; + case 78: goto st78; + case 79: goto st79; + case 80: goto st80; + case 81: goto st81; + case 82: goto st82; + case 83: goto st83; + case 84: goto st84; + case 85: goto st85; + case 86: goto st86; + case 87: goto st87; + case 88: goto st88; + case 89: goto st89; + case 90: goto st90; + case 91: goto st91; + case 92: goto st92; + case 93: goto st93; + case 94: goto st94; + case 95: goto st95; + case 96: goto st96; + case 97: goto st97; + case 16: goto st16; + case 17: goto st17; + case 98: goto st98; + case 18: goto st18; + case 19: goto st19; + case 99: goto st99; + case 20: goto st20; + case 21: goto st21; + case 22: goto st22; + case 100: goto st100; + case 101: goto st101; + case 23: goto st23; + case 102: goto st102; + case 103: goto st103; + case 104: goto st104; + case 105: goto st105; + case 106: goto st106; + case 107: goto st107; + case 108: goto st108; + case 109: goto st109; + case 110: goto st110; + case 111: goto st111; + case 112: goto st112; + case 113: goto st113; + case 114: goto st114; + case 115: goto st115; + case 116: goto st116; + case 117: goto st117; + case 118: goto st118; + case 119: goto st119; + case 120: goto st120; + case 121: goto st121; + case 122: goto st122; + case 123: goto st123; + case 124: goto st124; + case 125: goto st125; + case 126: goto st126; + case 127: goto st127; + case 128: goto st128; + case 129: goto st129; + case 130: goto st130; + case 131: goto st131; + case 132: goto st132; + case 133: goto st133; + case 134: goto st134; + case 135: goto st135; + case 136: goto st136; + case 137: goto st137; + case 138: goto st138; + case 139: goto st139; + case 140: goto st140; + case 141: goto st141; + case 142: goto st142; + case 143: goto st143; + case 0: goto st0; + case 144: goto st144; + case 145: goto st145; + case 146: goto st146; + case 147: goto st147; + case 148: goto st148; + case 24: goto st24; + case 149: goto st149; + case 25: goto st25; + case 150: goto st150; + case 26: goto st26; + case 151: goto st151; + case 152: goto st152; + case 153: goto st153; + case 27: goto st27; + case 28: goto st28; + case 154: goto st154; + case 155: goto st155; + case 156: goto st156; + case 157: goto st157; + case 158: goto st158; + case 29: goto st29; + case 159: goto st159; + case 160: goto st160; + case 161: goto st161; + case 162: goto st162; + case 163: goto st163; + case 164: goto st164; + case 165: goto st165; + case 166: goto st166; + case 167: goto st167; + case 168: goto st168; + case 169: goto st169; + case 170: goto st170; + case 171: goto st171; + case 172: goto st172; + case 173: goto st173; + case 174: goto st174; + case 175: goto st175; + case 176: goto st176; + case 177: goto st177; + case 178: goto st178; + case 179: goto st179; + case 180: goto st180; + case 181: goto st181; + case 182: goto st182; + case 183: goto st183; + case 184: goto st184; + case 185: goto st185; + case 186: goto st186; + case 187: goto st187; + case 188: goto st188; + case 189: goto st189; + case 190: goto st190; + case 191: goto st191; + case 192: goto st192; + case 193: goto st193; + case 194: goto st194; + case 195: goto st195; + case 196: goto st196; + case 197: goto st197; + case 198: goto st198; + case 199: goto st199; + case 200: goto st200; + case 201: goto st201; + case 202: goto st202; + case 203: goto st203; + case 204: goto st204; + case 205: goto st205; + case 206: goto st206; + case 207: goto st207; + case 208: goto st208; + case 209: goto st209; + case 210: goto st210; + case 211: goto st211; + case 212: goto st212; + case 213: goto st213; + case 214: goto st214; + case 215: goto st215; + case 216: goto st216; + case 217: goto st217; + case 218: goto st218; + case 219: goto st219; + case 220: goto st220; + case 221: goto st221; + case 222: goto st222; + case 223: goto st223; + case 224: goto st224; + case 225: goto st225; + case 226: goto st226; + case 227: goto st227; + case 228: goto st228; + case 229: goto st229; + case 230: goto st230; + case 231: goto st231; + case 232: goto st232; + case 233: goto st233; + case 234: goto st234; + case 235: goto st235; + case 236: goto st236; + case 237: goto st237; + case 238: goto st238; + case 239: goto st239; + case 240: goto st240; + case 241: goto st241; + case 242: goto st242; + case 243: goto st243; + case 244: goto st244; + case 245: goto st245; + case 246: goto st246; + case 247: goto st247; + case 248: goto st248; + case 249: goto st249; + case 250: goto st250; + case 251: goto st251; + case 252: goto st252; + case 30: goto st30; + case 253: goto st253; + case 254: goto st254; + case 255: goto st255; + case 31: goto st31; + case 32: goto st32; + case 256: goto st256; + case 33: goto st33; + case 257: goto st257; + case 258: goto st258; + case 259: goto st259; + case 34: goto st34; + case 35: goto st35; + case 260: goto st260; + case 36: goto st36; + case 37: goto st37; + case 261: goto st261; + case 262: goto st262; + default: break; + } + + if ( ++p == pe ) + goto _test_eof; +_resume: + switch ( cs ) + { +tr0: +#line 1171 "rlscan.rl" + {{p = ((te))-1;}{ pass( *ts, 0, 0 ); }} + goto st38; +tr3: +#line 1155 "rlscan.rl" + {te = p+1;{ pass( IMP_Literal, ts, te ); }} + goto st38; +tr11: +#line 1154 "rlscan.rl" + {te = p+1;{ pass(); }} + goto st38; +tr13: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 1154 "rlscan.rl" + {te = p+1;{ pass(); }} + goto st38; +tr71: +#line 1171 "rlscan.rl" + {te = p+1;{ pass( *ts, 0, 0 ); }} + goto st38; +tr72: +#line 1170 "rlscan.rl" + {te = p+1;} + goto st38; +tr82: +#line 1169 "rlscan.rl" + {te = p;p--;{ pass(); }} + goto st38; +tr83: +#line 1171 "rlscan.rl" + {te = p;p--;{ pass( *ts, 0, 0 ); }} + goto st38; +tr85: +#line 1163 "rlscan.rl" + {te = p;p--;{ + updateCol(); + singleLineSpec = true; + startSection(); + {stack[top++] = 38; goto st146;} + }} + goto st38; +tr86: +#line 1157 "rlscan.rl" + {te = p+1;{ + updateCol(); + singleLineSpec = false; + startSection(); + {stack[top++] = 38; goto st146;} + }} + goto st38; +tr87: +#line 1153 "rlscan.rl" + {te = p;p--;{ pass( IMP_UInt, ts, te ); }} + goto st38; +tr88: +#line 1 "NONE" + { switch( act ) { + case 176: + {{p = ((te))-1;} pass( IMP_Define, 0, 0 ); } + break; + case 177: + {{p = ((te))-1;} pass( IMP_Word, ts, te ); } + break; + } + } + goto st38; +tr89: +#line 1152 "rlscan.rl" + {te = p;p--;{ pass( IMP_Word, ts, te ); }} + goto st38; +st38: +#line 1 "NONE" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof38; +case 38: +#line 1 "NONE" + {ts = p;} +#line 1358 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr72; + case 9: goto st39; + case 10: goto tr74; + case 32: goto st39; + case 34: goto tr75; + case 37: goto st41; + case 39: goto tr77; + case 47: goto tr78; + case 95: goto tr80; + case 100: goto st47; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st45; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr71; +tr74: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st39; +st39: + if ( ++p == pe ) + goto _test_eof39; +case 39: +#line 1392 "rlscan.cpp" + switch( (*p) ) { + case 9: goto st39; + case 10: goto tr74; + case 32: goto st39; + } + goto tr82; +tr75: +#line 1 "NONE" + {te = p+1;} + goto st40; +st40: + if ( ++p == pe ) + goto _test_eof40; +case 40: +#line 1407 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr2; + case 34: goto tr3; + case 92: goto st2; + } + goto st1; +tr2: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st1; +st1: + if ( ++p == pe ) + goto _test_eof1; +case 1: +#line 1426 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr2; + case 34: goto tr3; + case 92: goto st2; + } + goto st1; +st2: + if ( ++p == pe ) + goto _test_eof2; +case 2: + if ( (*p) == 10 ) + goto tr2; + goto st1; +st41: + if ( ++p == pe ) + goto _test_eof41; +case 41: + if ( (*p) == 37 ) + goto st42; + goto tr83; +st42: + if ( ++p == pe ) + goto _test_eof42; +case 42: + if ( (*p) == 123 ) + goto tr86; + goto tr85; +tr77: +#line 1 "NONE" + {te = p+1;} + goto st43; +st43: + if ( ++p == pe ) + goto _test_eof43; +case 43: +#line 1462 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr6; + case 39: goto tr3; + case 92: goto st4; + } + goto st3; +tr6: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st3; +st3: + if ( ++p == pe ) + goto _test_eof3; +case 3: +#line 1481 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr6; + case 39: goto tr3; + case 92: goto st4; + } + goto st3; +st4: + if ( ++p == pe ) + goto _test_eof4; +case 4: + if ( (*p) == 10 ) + goto tr6; + goto st3; +tr78: +#line 1 "NONE" + {te = p+1;} + goto st44; +st44: + if ( ++p == pe ) + goto _test_eof44; +case 44: +#line 1503 "rlscan.cpp" + switch( (*p) ) { + case 42: goto st5; + case 47: goto st7; + } + goto tr83; +tr9: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st5; +st5: + if ( ++p == pe ) + goto _test_eof5; +case 5: +#line 1521 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr9; + case 42: goto st6; + } + goto st5; +st6: + if ( ++p == pe ) + goto _test_eof6; +case 6: + switch( (*p) ) { + case 10: goto tr9; + case 42: goto st6; + case 47: goto tr11; + } + goto st5; +st7: + if ( ++p == pe ) + goto _test_eof7; +case 7: + if ( (*p) == 10 ) + goto tr13; + goto st7; +st45: + if ( ++p == pe ) + goto _test_eof45; +case 45: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st45; + goto tr87; +tr80: +#line 1 "NONE" + {te = p+1;} +#line 1152 "rlscan.rl" + {act = 177;} + goto st46; +tr94: +#line 1 "NONE" + {te = p+1;} +#line 1151 "rlscan.rl" + {act = 176;} + goto st46; +st46: + if ( ++p == pe ) + goto _test_eof46; +case 46: +#line 1567 "rlscan.cpp" + if ( (*p) == 95 ) + goto tr80; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr88; +st47: + if ( ++p == pe ) + goto _test_eof47; +case 47: + switch( (*p) ) { + case 95: goto tr80; + case 101: goto st48; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr89; +st48: + if ( ++p == pe ) + goto _test_eof48; +case 48: + switch( (*p) ) { + case 95: goto tr80; + case 102: goto st49; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr89; +st49: + if ( ++p == pe ) + goto _test_eof49; +case 49: + switch( (*p) ) { + case 95: goto tr80; + case 105: goto st50; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr89; +st50: + if ( ++p == pe ) + goto _test_eof50; +case 50: + switch( (*p) ) { + case 95: goto tr80; + case 110: goto st51; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr89; +st51: + if ( ++p == pe ) + goto _test_eof51; +case 51: + switch( (*p) ) { + case 95: goto tr80; + case 101: goto tr94; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr80; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr80; + } else + goto tr80; + goto tr89; +tr14: +#line 770 "rlscan.rl" + {{p = ((te))-1;}{ token( IL_Symbol, ts, te ); }} + goto st52; +tr17: +#line 716 "rlscan.rl" + {te = p+1;{ token( IL_Literal, ts, te ); }} + goto st52; +tr20: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 723 "rlscan.rl" + {te = p+1;{ token( IL_Comment, ts, te ); }} + goto st52; +tr27: +#line 712 "rlscan.rl" + {{p = ((te))-1;}{ token( TK_UInt, ts, te ); }} + goto st52; +tr95: +#line 770 "rlscan.rl" + {te = p+1;{ token( IL_Symbol, ts, te ); }} + goto st52; +tr96: +#line 765 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated code block" << endl; + }} + goto st52; +tr102: +#line 745 "rlscan.rl" + {te = p+1;{ token( *ts, ts, te ); }} + goto st52; +tr103: +#line 740 "rlscan.rl" + {te = p+1;{ + whitespaceOn = true; + token( *ts, ts, te ); + }} + goto st52; +tr108: +#line 733 "rlscan.rl" + {te = p+1;{ + whitespaceOn = true; + token( *ts, ts, te ); + if ( inlineBlockType == SemiTerminated ) + {cs = stack[--top];goto _again;} + }} + goto st52; +tr111: +#line 747 "rlscan.rl" + {te = p+1;{ + token( IL_Symbol, ts, te ); + curly_count += 1; + }} + goto st52; +tr112: +#line 752 "rlscan.rl" + {te = p+1;{ + if ( --curly_count == 0 && inlineBlockType == CurlyDelimited ) { + /* Inline code block ends. */ + token( '}' ); + {cs = stack[--top];goto _again;} + } + else { + /* Either a semi terminated inline block or only the closing + * brace of some inner scope, not the block's closing brace. */ + token( IL_Symbol, ts, te ); + } + }} + goto st52; +tr113: +#line 718 "rlscan.rl" + {te = p;p--;{ + if ( whitespaceOn ) + token( IL_WhiteSpace, ts, te ); + }} + goto st52; +tr114: +#line 770 "rlscan.rl" + {te = p;p--;{ token( IL_Symbol, ts, te ); }} + goto st52; +tr115: +#line 712 "rlscan.rl" + {te = p;p--;{ token( TK_UInt, ts, te ); }} + goto st52; +tr117: +#line 713 "rlscan.rl" + {te = p;p--;{ token( TK_Hex, ts, te ); }} + goto st52; +tr118: +#line 725 "rlscan.rl" + {te = p+1;{ token( TK_NameSep, ts, te ); }} + goto st52; +tr119: +#line 1 "NONE" + { switch( act ) { + case 1: + {{p = ((te))-1;} token( KW_PChar ); } + break; + case 3: + {{p = ((te))-1;} token( KW_CurState ); } + break; + case 4: + {{p = ((te))-1;} token( KW_TargState ); } + break; + case 5: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Entry ); + } + break; + case 6: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Hold ); + } + break; + case 7: + {{p = ((te))-1;} token( KW_Exec, 0, 0 ); } + break; + case 8: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Goto ); + } + break; + case 9: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Next ); + } + break; + case 10: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Call ); + } + break; + case 11: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Ret ); + } + break; + case 12: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Break ); + } + break; + case 13: + {{p = ((te))-1;} token( TK_Word, ts, te ); } + break; + } + } + goto st52; +tr120: +#line 710 "rlscan.rl" + {te = p;p--;{ token( TK_Word, ts, te ); }} + goto st52; +tr134: +#line 675 "rlscan.rl" + {te = p;p--;{ token( KW_Char ); }} + goto st52; +st52: +#line 1 "NONE" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof52; +case 52: +#line 1 "NONE" + {ts = p;} +#line 1840 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr96; + case 9: goto st53; + case 10: goto tr98; + case 32: goto st53; + case 34: goto tr99; + case 35: goto tr100; + case 39: goto tr101; + case 40: goto tr102; + case 44: goto tr102; + case 47: goto tr104; + case 48: goto tr105; + case 58: goto st61; + case 59: goto tr108; + case 95: goto tr109; + case 102: goto st63; + case 123: goto tr111; + case 125: goto tr112; + } + if ( (*p) < 49 ) { + if ( 41 <= (*p) && (*p) <= 42 ) + goto tr103; + } else if ( (*p) > 57 ) { + if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else if ( (*p) >= 65 ) + goto tr109; + } else + goto st59; + goto tr95; +tr98: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st53; +st53: + if ( ++p == pe ) + goto _test_eof53; +case 53: +#line 1884 "rlscan.cpp" + switch( (*p) ) { + case 9: goto st53; + case 10: goto tr98; + case 32: goto st53; + } + goto tr113; +tr99: +#line 1 "NONE" + {te = p+1;} + goto st54; +st54: + if ( ++p == pe ) + goto _test_eof54; +case 54: +#line 1899 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr16; + case 34: goto tr17; + case 92: goto st9; + } + goto st8; +tr16: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st8; +st8: + if ( ++p == pe ) + goto _test_eof8; +case 8: +#line 1918 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr16; + case 34: goto tr17; + case 92: goto st9; + } + goto st8; +st9: + if ( ++p == pe ) + goto _test_eof9; +case 9: + if ( (*p) == 10 ) + goto tr16; + goto st8; +tr100: +#line 1 "NONE" + {te = p+1;} + goto st55; +st55: + if ( ++p == pe ) + goto _test_eof55; +case 55: +#line 1940 "rlscan.cpp" + if ( (*p) == 10 ) + goto tr20; + goto st10; +st10: + if ( ++p == pe ) + goto _test_eof10; +case 10: + if ( (*p) == 10 ) + goto tr20; + goto st10; +tr101: +#line 1 "NONE" + {te = p+1;} + goto st56; +st56: + if ( ++p == pe ) + goto _test_eof56; +case 56: +#line 1959 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr22; + case 39: goto tr17; + case 92: goto st12; + } + goto st11; +tr22: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st11; +st11: + if ( ++p == pe ) + goto _test_eof11; +case 11: +#line 1978 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr22; + case 39: goto tr17; + case 92: goto st12; + } + goto st11; +st12: + if ( ++p == pe ) + goto _test_eof12; +case 12: + if ( (*p) == 10 ) + goto tr22; + goto st11; +tr104: +#line 1 "NONE" + {te = p+1;} + goto st57; +st57: + if ( ++p == pe ) + goto _test_eof57; +case 57: +#line 2000 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr25; + case 47: goto tr17; + case 92: goto st14; + } + goto st13; +tr25: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st13; +st13: + if ( ++p == pe ) + goto _test_eof13; +case 13: +#line 2019 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr25; + case 47: goto tr17; + case 92: goto st14; + } + goto st13; +st14: + if ( ++p == pe ) + goto _test_eof14; +case 14: + if ( (*p) == 10 ) + goto tr25; + goto st13; +tr105: +#line 1 "NONE" + {te = p+1;} + goto st58; +st58: + if ( ++p == pe ) + goto _test_eof58; +case 58: +#line 2041 "rlscan.cpp" + if ( (*p) == 120 ) + goto st15; + if ( 48 <= (*p) && (*p) <= 57 ) + goto st59; + goto tr115; +st59: + if ( ++p == pe ) + goto _test_eof59; +case 59: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st59; + goto tr115; +st15: + if ( ++p == pe ) + goto _test_eof15; +case 15: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st60; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st60; + } else + goto st60; + goto tr27; +st60: + if ( ++p == pe ) + goto _test_eof60; +case 60: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st60; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st60; + } else + goto st60; + goto tr117; +st61: + if ( ++p == pe ) + goto _test_eof61; +case 61: + if ( (*p) == 58 ) + goto tr118; + goto tr114; +tr109: +#line 1 "NONE" + {te = p+1;} +#line 710 "rlscan.rl" + {act = 13;} + goto st62; +tr133: +#line 1 "NONE" + {te = p+1;} +#line 705 "rlscan.rl" + {act = 12;} + goto st62; +tr138: +#line 1 "NONE" + {te = p+1;} +#line 697 "rlscan.rl" + {act = 10;} + goto st62; +tr140: +#line 1 "NONE" + {te = p+1;} +#line 676 "rlscan.rl" + {act = 3;} + goto st62; +tr145: +#line 1 "NONE" + {te = p+1;} +#line 678 "rlscan.rl" + {act = 5;} + goto st62; +tr147: +#line 1 "NONE" + {te = p+1;} +#line 688 "rlscan.rl" + {act = 7;} + goto st62; +tr150: +#line 1 "NONE" + {te = p+1;} +#line 689 "rlscan.rl" + {act = 8;} + goto st62; +tr153: +#line 1 "NONE" + {te = p+1;} +#line 684 "rlscan.rl" + {act = 6;} + goto st62; +tr156: +#line 1 "NONE" + {te = p+1;} +#line 693 "rlscan.rl" + {act = 9;} + goto st62; +tr157: +#line 1 "NONE" + {te = p+1;} +#line 674 "rlscan.rl" + {act = 1;} + goto st62; +tr159: +#line 1 "NONE" + {te = p+1;} +#line 701 "rlscan.rl" + {act = 11;} + goto st62; +tr163: +#line 1 "NONE" + {te = p+1;} +#line 677 "rlscan.rl" + {act = 4;} + goto st62; +st62: + if ( ++p == pe ) + goto _test_eof62; +case 62: +#line 2163 "rlscan.cpp" + if ( (*p) == 95 ) + goto tr109; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr119; +st63: + if ( ++p == pe ) + goto _test_eof63; +case 63: + switch( (*p) ) { + case 95: goto tr109; + case 98: goto st64; + case 99: goto st68; + case 101: goto st73; + case 103: goto st79; + case 104: goto st82; + case 110: goto st85; + case 112: goto st88; + case 114: goto st89; + case 116: goto st91; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st64: + if ( ++p == pe ) + goto _test_eof64; +case 64: + switch( (*p) ) { + case 95: goto tr109; + case 114: goto st65; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st65: + if ( ++p == pe ) + goto _test_eof65; +case 65: + switch( (*p) ) { + case 95: goto tr109; + case 101: goto st66; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st66: + if ( ++p == pe ) + goto _test_eof66; +case 66: + switch( (*p) ) { + case 95: goto tr109; + case 97: goto st67; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st67: + if ( ++p == pe ) + goto _test_eof67; +case 67: + switch( (*p) ) { + case 95: goto tr109; + case 107: goto tr133; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st68: + if ( ++p == pe ) + goto _test_eof68; +case 68: + switch( (*p) ) { + case 95: goto tr109; + case 97: goto st69; + case 117: goto st71; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr134; +st69: + if ( ++p == pe ) + goto _test_eof69; +case 69: + switch( (*p) ) { + case 95: goto tr109; + case 108: goto st70; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st70: + if ( ++p == pe ) + goto _test_eof70; +case 70: + switch( (*p) ) { + case 95: goto tr109; + case 108: goto tr138; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st71: + if ( ++p == pe ) + goto _test_eof71; +case 71: + switch( (*p) ) { + case 95: goto tr109; + case 114: goto st72; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st72: + if ( ++p == pe ) + goto _test_eof72; +case 72: + switch( (*p) ) { + case 95: goto tr109; + case 115: goto tr140; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st73: + if ( ++p == pe ) + goto _test_eof73; +case 73: + switch( (*p) ) { + case 95: goto tr109; + case 110: goto st74; + case 120: goto st77; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st74: + if ( ++p == pe ) + goto _test_eof74; +case 74: + switch( (*p) ) { + case 95: goto tr109; + case 116: goto st75; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st75: + if ( ++p == pe ) + goto _test_eof75; +case 75: + switch( (*p) ) { + case 95: goto tr109; + case 114: goto st76; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st76: + if ( ++p == pe ) + goto _test_eof76; +case 76: + switch( (*p) ) { + case 95: goto tr109; + case 121: goto tr145; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st77: + if ( ++p == pe ) + goto _test_eof77; +case 77: + switch( (*p) ) { + case 95: goto tr109; + case 101: goto st78; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st78: + if ( ++p == pe ) + goto _test_eof78; +case 78: + switch( (*p) ) { + case 95: goto tr109; + case 99: goto tr147; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st79: + if ( ++p == pe ) + goto _test_eof79; +case 79: + switch( (*p) ) { + case 95: goto tr109; + case 111: goto st80; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st80: + if ( ++p == pe ) + goto _test_eof80; +case 80: + switch( (*p) ) { + case 95: goto tr109; + case 116: goto st81; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st81: + if ( ++p == pe ) + goto _test_eof81; +case 81: + switch( (*p) ) { + case 95: goto tr109; + case 111: goto tr150; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st82: + if ( ++p == pe ) + goto _test_eof82; +case 82: + switch( (*p) ) { + case 95: goto tr109; + case 111: goto st83; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st83: + if ( ++p == pe ) + goto _test_eof83; +case 83: + switch( (*p) ) { + case 95: goto tr109; + case 108: goto st84; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st84: + if ( ++p == pe ) + goto _test_eof84; +case 84: + switch( (*p) ) { + case 95: goto tr109; + case 100: goto tr153; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st85: + if ( ++p == pe ) + goto _test_eof85; +case 85: + switch( (*p) ) { + case 95: goto tr109; + case 101: goto st86; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st86: + if ( ++p == pe ) + goto _test_eof86; +case 86: + switch( (*p) ) { + case 95: goto tr109; + case 120: goto st87; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st87: + if ( ++p == pe ) + goto _test_eof87; +case 87: + switch( (*p) ) { + case 95: goto tr109; + case 116: goto tr156; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st88: + if ( ++p == pe ) + goto _test_eof88; +case 88: + switch( (*p) ) { + case 95: goto tr109; + case 99: goto tr157; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st89: + if ( ++p == pe ) + goto _test_eof89; +case 89: + switch( (*p) ) { + case 95: goto tr109; + case 101: goto st90; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st90: + if ( ++p == pe ) + goto _test_eof90; +case 90: + switch( (*p) ) { + case 95: goto tr109; + case 116: goto tr159; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st91: + if ( ++p == pe ) + goto _test_eof91; +case 91: + switch( (*p) ) { + case 95: goto tr109; + case 97: goto st92; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st92: + if ( ++p == pe ) + goto _test_eof92; +case 92: + switch( (*p) ) { + case 95: goto tr109; + case 114: goto st93; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st93: + if ( ++p == pe ) + goto _test_eof93; +case 93: + switch( (*p) ) { + case 95: goto tr109; + case 103: goto st94; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +st94: + if ( ++p == pe ) + goto _test_eof94; +case 94: + switch( (*p) ) { + case 95: goto tr109; + case 115: goto tr163; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr109; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr109; + } else + goto tr109; + goto tr120; +tr29: +#line 873 "rlscan.rl" + {{p = ((te))-1;}{ token( IL_Symbol, ts, te ); }} + goto st95; +tr32: +#line 819 "rlscan.rl" + {te = p+1;{ token( IL_Literal, ts, te ); }} + goto st95; +tr40: +#line 826 "rlscan.rl" + {te = p+1;{ token( IL_Comment, ts, te ); }} + goto st95; +tr42: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 826 "rlscan.rl" + {te = p+1;{ token( IL_Comment, ts, te ); }} + goto st95; +tr43: +#line 815 "rlscan.rl" + {{p = ((te))-1;}{ token( TK_UInt, ts, te ); }} + goto st95; +tr164: +#line 873 "rlscan.rl" + {te = p+1;{ token( IL_Symbol, ts, te ); }} + goto st95; +tr165: +#line 868 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated code block" << endl; + }} + goto st95; +tr170: +#line 848 "rlscan.rl" + {te = p+1;{ token( *ts, ts, te ); }} + goto st95; +tr171: +#line 843 "rlscan.rl" + {te = p+1;{ + whitespaceOn = true; + token( *ts, ts, te ); + }} + goto st95; +tr176: +#line 836 "rlscan.rl" + {te = p+1;{ + whitespaceOn = true; + token( *ts, ts, te ); + if ( inlineBlockType == SemiTerminated ) + {cs = stack[--top];goto _again;} + }} + goto st95; +tr179: +#line 850 "rlscan.rl" + {te = p+1;{ + token( IL_Symbol, ts, te ); + curly_count += 1; + }} + goto st95; +tr180: +#line 855 "rlscan.rl" + {te = p+1;{ + if ( --curly_count == 0 && inlineBlockType == CurlyDelimited ) { + /* Inline code block ends. */ + token( '}' ); + {cs = stack[--top];goto _again;} + } + else { + /* Either a semi terminated inline block or only the closing + * brace of some inner scope, not the block's closing brace. */ + token( IL_Symbol, ts, te ); + } + }} + goto st95; +tr181: +#line 821 "rlscan.rl" + {te = p;p--;{ + if ( whitespaceOn ) + token( IL_WhiteSpace, ts, te ); + }} + goto st95; +tr182: +#line 873 "rlscan.rl" + {te = p;p--;{ token( IL_Symbol, ts, te ); }} + goto st95; +tr183: +#line 815 "rlscan.rl" + {te = p;p--;{ token( TK_UInt, ts, te ); }} + goto st95; +tr185: +#line 816 "rlscan.rl" + {te = p;p--;{ token( TK_Hex, ts, te ); }} + goto st95; +tr186: +#line 828 "rlscan.rl" + {te = p+1;{ token( TK_NameSep, ts, te ); }} + goto st95; +tr187: +#line 1 "NONE" + { switch( act ) { + case 27: + {{p = ((te))-1;} token( KW_PChar ); } + break; + case 29: + {{p = ((te))-1;} token( KW_CurState ); } + break; + case 30: + {{p = ((te))-1;} token( KW_TargState ); } + break; + case 31: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Entry ); + } + break; + case 32: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Hold ); + } + break; + case 33: + {{p = ((te))-1;} token( KW_Exec, 0, 0 ); } + break; + case 34: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Goto ); + } + break; + case 35: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Next ); + } + break; + case 36: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Call ); + } + break; + case 37: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Ret ); + } + break; + case 38: + {{p = ((te))-1;} + whitespaceOn = false; + token( KW_Break ); + } + break; + case 39: + {{p = ((te))-1;} token( TK_Word, ts, te ); } + break; + } + } + goto st95; +tr188: +#line 813 "rlscan.rl" + {te = p;p--;{ token( TK_Word, ts, te ); }} + goto st95; +tr202: +#line 778 "rlscan.rl" + {te = p;p--;{ token( KW_Char ); }} + goto st95; +st95: +#line 1 "NONE" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof95; +case 95: +#line 1 "NONE" + {ts = p;} +#line 2909 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr165; + case 9: goto st96; + case 10: goto tr167; + case 32: goto st96; + case 34: goto tr168; + case 39: goto tr169; + case 40: goto tr170; + case 44: goto tr170; + case 47: goto tr172; + case 48: goto tr173; + case 58: goto st103; + case 59: goto tr176; + case 95: goto tr177; + case 102: goto st105; + case 123: goto tr179; + case 125: goto tr180; + } + if ( (*p) < 49 ) { + if ( 41 <= (*p) && (*p) <= 42 ) + goto tr171; + } else if ( (*p) > 57 ) { + if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else if ( (*p) >= 65 ) + goto tr177; + } else + goto st101; + goto tr164; +tr167: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st96; +st96: + if ( ++p == pe ) + goto _test_eof96; +case 96: +#line 2952 "rlscan.cpp" + switch( (*p) ) { + case 9: goto st96; + case 10: goto tr167; + case 32: goto st96; + } + goto tr181; +tr168: +#line 1 "NONE" + {te = p+1;} + goto st97; +st97: + if ( ++p == pe ) + goto _test_eof97; +case 97: +#line 2967 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr31; + case 34: goto tr32; + case 92: goto st17; + } + goto st16; +tr31: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st16; +st16: + if ( ++p == pe ) + goto _test_eof16; +case 16: +#line 2986 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr31; + case 34: goto tr32; + case 92: goto st17; + } + goto st16; +st17: + if ( ++p == pe ) + goto _test_eof17; +case 17: + if ( (*p) == 10 ) + goto tr31; + goto st16; +tr169: +#line 1 "NONE" + {te = p+1;} + goto st98; +st98: + if ( ++p == pe ) + goto _test_eof98; +case 98: +#line 3008 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr35; + case 39: goto tr32; + case 92: goto st19; + } + goto st18; +tr35: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st18; +st18: + if ( ++p == pe ) + goto _test_eof18; +case 18: +#line 3027 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr35; + case 39: goto tr32; + case 92: goto st19; + } + goto st18; +st19: + if ( ++p == pe ) + goto _test_eof19; +case 19: + if ( (*p) == 10 ) + goto tr35; + goto st18; +tr172: +#line 1 "NONE" + {te = p+1;} + goto st99; +st99: + if ( ++p == pe ) + goto _test_eof99; +case 99: +#line 3049 "rlscan.cpp" + switch( (*p) ) { + case 42: goto st20; + case 47: goto st22; + } + goto tr182; +tr38: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st20; +st20: + if ( ++p == pe ) + goto _test_eof20; +case 20: +#line 3067 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr38; + case 42: goto st21; + } + goto st20; +st21: + if ( ++p == pe ) + goto _test_eof21; +case 21: + switch( (*p) ) { + case 10: goto tr38; + case 42: goto st21; + case 47: goto tr40; + } + goto st20; +st22: + if ( ++p == pe ) + goto _test_eof22; +case 22: + if ( (*p) == 10 ) + goto tr42; + goto st22; +tr173: +#line 1 "NONE" + {te = p+1;} + goto st100; +st100: + if ( ++p == pe ) + goto _test_eof100; +case 100: +#line 3098 "rlscan.cpp" + if ( (*p) == 120 ) + goto st23; + if ( 48 <= (*p) && (*p) <= 57 ) + goto st101; + goto tr183; +st101: + if ( ++p == pe ) + goto _test_eof101; +case 101: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st101; + goto tr183; +st23: + if ( ++p == pe ) + goto _test_eof23; +case 23: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st102; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st102; + } else + goto st102; + goto tr43; +st102: + if ( ++p == pe ) + goto _test_eof102; +case 102: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st102; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st102; + } else + goto st102; + goto tr185; +st103: + if ( ++p == pe ) + goto _test_eof103; +case 103: + if ( (*p) == 58 ) + goto tr186; + goto tr182; +tr177: +#line 1 "NONE" + {te = p+1;} +#line 813 "rlscan.rl" + {act = 39;} + goto st104; +tr201: +#line 1 "NONE" + {te = p+1;} +#line 808 "rlscan.rl" + {act = 38;} + goto st104; +tr206: +#line 1 "NONE" + {te = p+1;} +#line 800 "rlscan.rl" + {act = 36;} + goto st104; +tr208: +#line 1 "NONE" + {te = p+1;} +#line 779 "rlscan.rl" + {act = 29;} + goto st104; +tr213: +#line 1 "NONE" + {te = p+1;} +#line 781 "rlscan.rl" + {act = 31;} + goto st104; +tr215: +#line 1 "NONE" + {te = p+1;} +#line 791 "rlscan.rl" + {act = 33;} + goto st104; +tr218: +#line 1 "NONE" + {te = p+1;} +#line 792 "rlscan.rl" + {act = 34;} + goto st104; +tr221: +#line 1 "NONE" + {te = p+1;} +#line 787 "rlscan.rl" + {act = 32;} + goto st104; +tr224: +#line 1 "NONE" + {te = p+1;} +#line 796 "rlscan.rl" + {act = 35;} + goto st104; +tr225: +#line 1 "NONE" + {te = p+1;} +#line 777 "rlscan.rl" + {act = 27;} + goto st104; +tr227: +#line 1 "NONE" + {te = p+1;} +#line 804 "rlscan.rl" + {act = 37;} + goto st104; +tr231: +#line 1 "NONE" + {te = p+1;} +#line 780 "rlscan.rl" + {act = 30;} + goto st104; +st104: + if ( ++p == pe ) + goto _test_eof104; +case 104: +#line 3220 "rlscan.cpp" + if ( (*p) == 95 ) + goto tr177; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr187; +st105: + if ( ++p == pe ) + goto _test_eof105; +case 105: + switch( (*p) ) { + case 95: goto tr177; + case 98: goto st106; + case 99: goto st110; + case 101: goto st115; + case 103: goto st121; + case 104: goto st124; + case 110: goto st127; + case 112: goto st130; + case 114: goto st131; + case 116: goto st133; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st106: + if ( ++p == pe ) + goto _test_eof106; +case 106: + switch( (*p) ) { + case 95: goto tr177; + case 114: goto st107; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st107: + if ( ++p == pe ) + goto _test_eof107; +case 107: + switch( (*p) ) { + case 95: goto tr177; + case 101: goto st108; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st108: + if ( ++p == pe ) + goto _test_eof108; +case 108: + switch( (*p) ) { + case 95: goto tr177; + case 97: goto st109; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st109: + if ( ++p == pe ) + goto _test_eof109; +case 109: + switch( (*p) ) { + case 95: goto tr177; + case 107: goto tr201; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st110: + if ( ++p == pe ) + goto _test_eof110; +case 110: + switch( (*p) ) { + case 95: goto tr177; + case 97: goto st111; + case 117: goto st113; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr202; +st111: + if ( ++p == pe ) + goto _test_eof111; +case 111: + switch( (*p) ) { + case 95: goto tr177; + case 108: goto st112; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st112: + if ( ++p == pe ) + goto _test_eof112; +case 112: + switch( (*p) ) { + case 95: goto tr177; + case 108: goto tr206; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st113: + if ( ++p == pe ) + goto _test_eof113; +case 113: + switch( (*p) ) { + case 95: goto tr177; + case 114: goto st114; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st114: + if ( ++p == pe ) + goto _test_eof114; +case 114: + switch( (*p) ) { + case 95: goto tr177; + case 115: goto tr208; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st115: + if ( ++p == pe ) + goto _test_eof115; +case 115: + switch( (*p) ) { + case 95: goto tr177; + case 110: goto st116; + case 120: goto st119; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st116: + if ( ++p == pe ) + goto _test_eof116; +case 116: + switch( (*p) ) { + case 95: goto tr177; + case 116: goto st117; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st117: + if ( ++p == pe ) + goto _test_eof117; +case 117: + switch( (*p) ) { + case 95: goto tr177; + case 114: goto st118; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st118: + if ( ++p == pe ) + goto _test_eof118; +case 118: + switch( (*p) ) { + case 95: goto tr177; + case 121: goto tr213; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st119: + if ( ++p == pe ) + goto _test_eof119; +case 119: + switch( (*p) ) { + case 95: goto tr177; + case 101: goto st120; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st120: + if ( ++p == pe ) + goto _test_eof120; +case 120: + switch( (*p) ) { + case 95: goto tr177; + case 99: goto tr215; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st121: + if ( ++p == pe ) + goto _test_eof121; +case 121: + switch( (*p) ) { + case 95: goto tr177; + case 111: goto st122; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st122: + if ( ++p == pe ) + goto _test_eof122; +case 122: + switch( (*p) ) { + case 95: goto tr177; + case 116: goto st123; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st123: + if ( ++p == pe ) + goto _test_eof123; +case 123: + switch( (*p) ) { + case 95: goto tr177; + case 111: goto tr218; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st124: + if ( ++p == pe ) + goto _test_eof124; +case 124: + switch( (*p) ) { + case 95: goto tr177; + case 111: goto st125; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st125: + if ( ++p == pe ) + goto _test_eof125; +case 125: + switch( (*p) ) { + case 95: goto tr177; + case 108: goto st126; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st126: + if ( ++p == pe ) + goto _test_eof126; +case 126: + switch( (*p) ) { + case 95: goto tr177; + case 100: goto tr221; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st127: + if ( ++p == pe ) + goto _test_eof127; +case 127: + switch( (*p) ) { + case 95: goto tr177; + case 101: goto st128; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st128: + if ( ++p == pe ) + goto _test_eof128; +case 128: + switch( (*p) ) { + case 95: goto tr177; + case 120: goto st129; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st129: + if ( ++p == pe ) + goto _test_eof129; +case 129: + switch( (*p) ) { + case 95: goto tr177; + case 116: goto tr224; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st130: + if ( ++p == pe ) + goto _test_eof130; +case 130: + switch( (*p) ) { + case 95: goto tr177; + case 99: goto tr225; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st131: + if ( ++p == pe ) + goto _test_eof131; +case 131: + switch( (*p) ) { + case 95: goto tr177; + case 101: goto st132; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st132: + if ( ++p == pe ) + goto _test_eof132; +case 132: + switch( (*p) ) { + case 95: goto tr177; + case 116: goto tr227; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st133: + if ( ++p == pe ) + goto _test_eof133; +case 133: + switch( (*p) ) { + case 95: goto tr177; + case 97: goto st134; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st134: + if ( ++p == pe ) + goto _test_eof134; +case 134: + switch( (*p) ) { + case 95: goto tr177; + case 114: goto st135; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st135: + if ( ++p == pe ) + goto _test_eof135; +case 135: + switch( (*p) ) { + case 95: goto tr177; + case 103: goto st136; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +st136: + if ( ++p == pe ) + goto _test_eof136; +case 136: + switch( (*p) ) { + case 95: goto tr177; + case 115: goto tr231; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr177; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr177; + } else + goto tr177; + goto tr188; +tr232: +#line 900 "rlscan.rl" + {te = p+1;{ token( RE_Char, ts, te ); }} + goto st137; +tr233: +#line 895 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated OR literal" << endl; + }} + goto st137; +tr234: +#line 890 "rlscan.rl" + {te = p+1;{ token( RE_Dash, 0, 0 ); }} + goto st137; +tr236: +#line 893 "rlscan.rl" + {te = p+1;{ token( RE_SqClose ); {cs = stack[--top];goto _again;} }} + goto st137; +tr237: +#line 900 "rlscan.rl" + {te = p;p--;{ token( RE_Char, ts, te ); }} + goto st137; +tr238: +#line 887 "rlscan.rl" + {te = p+1;{ token( RE_Char, ts+1, te ); }} + goto st137; +tr239: +#line 886 "rlscan.rl" + {te = p+1;{ updateCol(); }} + goto st137; +tr240: +#line 878 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\0' ); }} + goto st137; +tr241: +#line 879 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\a' ); }} + goto st137; +tr242: +#line 880 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\b' ); }} + goto st137; +tr243: +#line 884 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\f' ); }} + goto st137; +tr244: +#line 882 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\n' ); }} + goto st137; +tr245: +#line 885 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\r' ); }} + goto st137; +tr246: +#line 881 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\t' ); }} + goto st137; +tr247: +#line 883 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\v' ); }} + goto st137; +st137: +#line 1 "NONE" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof137; +case 137: +#line 1 "NONE" + {ts = p;} +#line 3856 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr233; + case 45: goto tr234; + case 92: goto st138; + case 93: goto tr236; + } + goto tr232; +st138: + if ( ++p == pe ) + goto _test_eof138; +case 138: + switch( (*p) ) { + case 10: goto tr239; + case 48: goto tr240; + case 97: goto tr241; + case 98: goto tr242; + case 102: goto tr243; + case 110: goto tr244; + case 114: goto tr245; + case 116: goto tr246; + case 118: goto tr247; + } + goto tr238; +tr248: +#line 935 "rlscan.rl" + {te = p+1;{ token( RE_Char, ts, te ); }} + goto st139; +tr249: +#line 930 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated regular expression" << endl; + }} + goto st139; +tr250: +#line 925 "rlscan.rl" + {te = p+1;{ token( RE_Star ); }} + goto st139; +tr251: +#line 924 "rlscan.rl" + {te = p+1;{ token( RE_Dot ); }} + goto st139; +tr255: +#line 918 "rlscan.rl" + {te = p;p--;{ + token( RE_Slash, ts, te ); + {goto st146;} + }} + goto st139; +tr256: +#line 918 "rlscan.rl" + {te = p+1;{ + token( RE_Slash, ts, te ); + {goto st146;} + }} + goto st139; +tr257: +#line 927 "rlscan.rl" + {te = p;p--;{ token( RE_SqOpen ); {stack[top++] = 139; goto st137;} }} + goto st139; +tr258: +#line 928 "rlscan.rl" + {te = p+1;{ token( RE_SqOpenNeg ); {stack[top++] = 139; goto st137;} }} + goto st139; +tr259: +#line 935 "rlscan.rl" + {te = p;p--;{ token( RE_Char, ts, te ); }} + goto st139; +tr260: +#line 915 "rlscan.rl" + {te = p+1;{ token( RE_Char, ts+1, te ); }} + goto st139; +tr261: +#line 914 "rlscan.rl" + {te = p+1;{ updateCol(); }} + goto st139; +tr262: +#line 906 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\0' ); }} + goto st139; +tr263: +#line 907 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\a' ); }} + goto st139; +tr264: +#line 908 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\b' ); }} + goto st139; +tr265: +#line 912 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\f' ); }} + goto st139; +tr266: +#line 910 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\n' ); }} + goto st139; +tr267: +#line 913 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\r' ); }} + goto st139; +tr268: +#line 909 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\t' ); }} + goto st139; +tr269: +#line 911 "rlscan.rl" + {te = p+1;{ token( RE_Char, '\v' ); }} + goto st139; +st139: +#line 1 "NONE" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof139; +case 139: +#line 1 "NONE" + {ts = p;} +#line 3972 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr249; + case 42: goto tr250; + case 46: goto tr251; + case 47: goto st140; + case 91: goto st141; + case 92: goto st142; + } + goto tr248; +st140: + if ( ++p == pe ) + goto _test_eof140; +case 140: + if ( (*p) == 105 ) + goto tr256; + goto tr255; +st141: + if ( ++p == pe ) + goto _test_eof141; +case 141: + if ( (*p) == 94 ) + goto tr258; + goto tr257; +st142: + if ( ++p == pe ) + goto _test_eof142; +case 142: + switch( (*p) ) { + case 10: goto tr261; + case 48: goto tr262; + case 97: goto tr263; + case 98: goto tr264; + case 102: goto tr265; + case 110: goto tr266; + case 114: goto tr267; + case 116: goto tr268; + case 118: goto tr269; + } + goto tr260; +tr270: +#line 944 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated write statement" << endl; + }} + goto st143; +tr273: +#line 942 "rlscan.rl" + {te = p+1;{ token( ';' ); {goto st146;} }} + goto st143; +tr275: +#line 941 "rlscan.rl" + {te = p;p--;{ updateCol(); }} + goto st143; +tr276: +#line 940 "rlscan.rl" + {te = p;p--;{ token( TK_Word, ts, te ); }} + goto st143; +st143: +#line 1 "NONE" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof143; +case 143: +#line 1 "NONE" + {ts = p;} +#line 4038 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr270; + case 32: goto st144; + case 59: goto tr273; + case 95: goto st145; + } + if ( (*p) < 65 ) { + if ( 9 <= (*p) && (*p) <= 10 ) + goto st144; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto st145; + } else + goto st145; + goto st0; +st0: +cs = 0; + goto _out; +st144: + if ( ++p == pe ) + goto _test_eof144; +case 144: + if ( (*p) == 32 ) + goto st144; + if ( 9 <= (*p) && (*p) <= 10 ) + goto st144; + goto tr275; +st145: + if ( ++p == pe ) + goto _test_eof145; +case 145: + if ( (*p) == 95 ) + goto st145; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st145; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto st145; + } else + goto st145; + goto tr276; +tr45: +#line 1121 "rlscan.rl" + {{p = ((te))-1;}{ token( *ts ); }} + goto st146; +tr51: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 1018 "rlscan.rl" + {te = p+1;{ updateCol(); }} + goto st146; +tr55: +#line 1005 "rlscan.rl" + {{p = ((te))-1;}{ token( TK_UInt, ts, te ); }} + goto st146; +tr57: +#line 1086 "rlscan.rl" + {te = p+1;{ + updateCol(); + endSection(); + {cs = stack[--top];goto _again;} + }} + goto st146; +tr277: +#line 1121 "rlscan.rl" + {te = p+1;{ token( *ts ); }} + goto st146; +tr278: +#line 1117 "rlscan.rl" + {te = p+1;{ + scan_error() << "unterminated ragel section" << endl; + }} + goto st146; +tr280: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 1095 "rlscan.rl" + {te = p+1;{ + updateCol(); + if ( singleLineSpec ) { + endSection(); + {cs = stack[--top];goto _again;} + } + }} + goto st146; +tr289: +#line 1015 "rlscan.rl" + {te = p+1;{ token( RE_Slash ); {goto st139;} }} + goto st146; +tr311: +#line 1103 "rlscan.rl" + {te = p+1;{ + if ( lastToken == KW_Export || lastToken == KW_Entry ) + token( '{' ); + else { + token( '{' ); + curly_count = 1; + inlineBlockType = CurlyDelimited; + if ( hostLang->lang == HostLang::Ruby ) + {stack[top++] = 146; goto st52;} + else + {stack[top++] = 146; goto st95;} + } + }} + goto st146; +tr314: +#line 1092 "rlscan.rl" + {te = p;p--;{ updateCol(); }} + goto st146; +tr315: +#line 1121 "rlscan.rl" + {te = p;p--;{ token( *ts ); }} + goto st146; +tr316: +#line 1010 "rlscan.rl" + {te = p;p--;{ token( TK_Literal, ts, te ); }} + goto st146; +tr317: +#line 1010 "rlscan.rl" + {te = p+1;{ token( TK_Literal, ts, te ); }} + goto st146; +tr318: +#line 1048 "rlscan.rl" + {te = p+1;{ token( TK_AllGblError ); }} + goto st146; +tr319: +#line 1032 "rlscan.rl" + {te = p+1;{ token( TK_AllFromState ); }} + goto st146; +tr320: +#line 1040 "rlscan.rl" + {te = p+1;{ token( TK_AllEOF ); }} + goto st146; +tr321: +#line 1067 "rlscan.rl" + {te = p+1;{ token( TK_AllCond ); }} + goto st146; +tr322: +#line 1056 "rlscan.rl" + {te = p+1;{ token( TK_AllLocalError ); }} + goto st146; +tr323: +#line 1024 "rlscan.rl" + {te = p+1;{ token( TK_AllToState ); }} + goto st146; +tr324: +#line 1049 "rlscan.rl" + {te = p+1;{ token( TK_FinalGblError ); }} + goto st146; +tr325: +#line 1033 "rlscan.rl" + {te = p+1;{ token( TK_FinalFromState ); }} + goto st146; +tr326: +#line 1041 "rlscan.rl" + {te = p+1;{ token( TK_FinalEOF ); }} + goto st146; +tr327: +#line 1068 "rlscan.rl" + {te = p+1;{ token( TK_LeavingCond ); }} + goto st146; +tr328: +#line 1057 "rlscan.rl" + {te = p+1;{ token( TK_FinalLocalError ); }} + goto st146; +tr329: +#line 1025 "rlscan.rl" + {te = p+1;{ token( TK_FinalToState ); }} + goto st146; +tr330: +#line 1071 "rlscan.rl" + {te = p+1;{ token( TK_StarStar ); }} + goto st146; +tr331: +#line 1072 "rlscan.rl" + {te = p+1;{ token( TK_DashDash ); }} + goto st146; +tr332: +#line 1073 "rlscan.rl" + {te = p+1;{ token( TK_Arrow ); }} + goto st146; +tr333: +#line 1070 "rlscan.rl" + {te = p+1;{ token( TK_DotDot ); }} + goto st146; +tr334: +#line 1005 "rlscan.rl" + {te = p;p--;{ token( TK_UInt, ts, te ); }} + goto st146; +tr336: +#line 1006 "rlscan.rl" + {te = p;p--;{ token( TK_Hex, ts, te ); }} + goto st146; +tr337: +#line 1084 "rlscan.rl" + {te = p+1;{ token( TK_NameSep, ts, te ); }} + goto st146; +tr338: +#line 1020 "rlscan.rl" + {te = p+1;{ token( TK_ColonEquals ); }} + goto st146; +tr340: +#line 1076 "rlscan.rl" + {te = p;p--;{ token( TK_ColonGt ); }} + goto st146; +tr341: +#line 1077 "rlscan.rl" + {te = p+1;{ token( TK_ColonGtGt ); }} + goto st146; +tr342: +#line 1050 "rlscan.rl" + {te = p+1;{ token( TK_NotStartGblError ); }} + goto st146; +tr343: +#line 1034 "rlscan.rl" + {te = p+1;{ token( TK_NotStartFromState ); }} + goto st146; +tr344: +#line 1042 "rlscan.rl" + {te = p+1;{ token( TK_NotStartEOF ); }} + goto st146; +tr345: +#line 1078 "rlscan.rl" + {te = p+1;{ token( TK_LtColon ); }} + goto st146; +tr347: +#line 1058 "rlscan.rl" + {te = p+1;{ token( TK_NotStartLocalError ); }} + goto st146; +tr348: +#line 1026 "rlscan.rl" + {te = p+1;{ token( TK_NotStartToState ); }} + goto st146; +tr349: +#line 1063 "rlscan.rl" + {te = p;p--;{ token( TK_Middle ); }} + goto st146; +tr350: +#line 1052 "rlscan.rl" + {te = p+1;{ token( TK_MiddleGblError ); }} + goto st146; +tr351: +#line 1036 "rlscan.rl" + {te = p+1;{ token( TK_MiddleFromState ); }} + goto st146; +tr352: +#line 1044 "rlscan.rl" + {te = p+1;{ token( TK_MiddleEOF ); }} + goto st146; +tr353: +#line 1060 "rlscan.rl" + {te = p+1;{ token( TK_MiddleLocalError ); }} + goto st146; +tr354: +#line 1028 "rlscan.rl" + {te = p+1;{ token( TK_MiddleToState ); }} + goto st146; +tr355: +#line 1074 "rlscan.rl" + {te = p+1;{ token( TK_DoubleArrow ); }} + goto st146; +tr356: +#line 1047 "rlscan.rl" + {te = p+1;{ token( TK_StartGblError ); }} + goto st146; +tr357: +#line 1031 "rlscan.rl" + {te = p+1;{ token( TK_StartFromState ); }} + goto st146; +tr358: +#line 1039 "rlscan.rl" + {te = p+1;{ token( TK_StartEOF ); }} + goto st146; +tr359: +#line 1066 "rlscan.rl" + {te = p+1;{ token( TK_StartCond ); }} + goto st146; +tr360: +#line 1055 "rlscan.rl" + {te = p+1;{ token( TK_StartLocalError ); }} + goto st146; +tr361: +#line 1023 "rlscan.rl" + {te = p+1;{ token( TK_StartToState ); }} + goto st146; +tr362: +#line 1051 "rlscan.rl" + {te = p+1;{ token( TK_NotFinalGblError ); }} + goto st146; +tr363: +#line 1035 "rlscan.rl" + {te = p+1;{ token( TK_NotFinalFromState ); }} + goto st146; +tr364: +#line 1043 "rlscan.rl" + {te = p+1;{ token( TK_NotFinalEOF ); }} + goto st146; +tr365: +#line 1059 "rlscan.rl" + {te = p+1;{ token( TK_NotFinalLocalError ); }} + goto st146; +tr366: +#line 1027 "rlscan.rl" + {te = p+1;{ token( TK_NotFinalToState ); }} + goto st146; +tr367: +#line 1 "NONE" + { switch( act ) { + case 88: + {{p = ((te))-1;} token( KW_Machine ); } + break; + case 89: + {{p = ((te))-1;} token( KW_Include ); } + break; + case 90: + {{p = ((te))-1;} token( KW_Import ); } + break; + case 91: + {{p = ((te))-1;} + token( KW_Write ); + {goto st143;} + } + break; + case 92: + {{p = ((te))-1;} token( KW_Action ); } + break; + case 93: + {{p = ((te))-1;} token( KW_AlphType ); } + break; + case 94: + {{p = ((te))-1;} token( KW_PrePush ); } + break; + case 95: + {{p = ((te))-1;} token( KW_PostPop ); } + break; + case 96: + {{p = ((te))-1;} + token( KW_GetKey ); + inlineBlockType = SemiTerminated; + if ( hostLang->lang == HostLang::Ruby ) + {stack[top++] = 146; goto st52;} + else + {stack[top++] = 146; goto st95;} + } + break; + case 97: + {{p = ((te))-1;} + token( KW_Access ); + inlineBlockType = SemiTerminated; + if ( hostLang->lang == HostLang::Ruby ) + {stack[top++] = 146; goto st52;} + else + {stack[top++] = 146; goto st95;} + } + break; + case 98: + {{p = ((te))-1;} + token( KW_Variable ); + inlineBlockType = SemiTerminated; + if ( hostLang->lang == HostLang::Ruby ) + {stack[top++] = 146; goto st52;} + else + {stack[top++] = 146; goto st95;} + } + break; + case 99: + {{p = ((te))-1;} token( KW_When ); } + break; + case 100: + {{p = ((te))-1;} token( KW_InWhen ); } + break; + case 101: + {{p = ((te))-1;} token( KW_OutWhen ); } + break; + case 102: + {{p = ((te))-1;} token( KW_Eof ); } + break; + case 103: + {{p = ((te))-1;} token( KW_Err ); } + break; + case 104: + {{p = ((te))-1;} token( KW_Lerr ); } + break; + case 105: + {{p = ((te))-1;} token( KW_To ); } + break; + case 106: + {{p = ((te))-1;} token( KW_From ); } + break; + case 107: + {{p = ((te))-1;} token( KW_Export ); } + break; + case 108: + {{p = ((te))-1;} token( TK_Word, ts, te ); } + break; + } + } + goto st146; +tr368: +#line 1012 "rlscan.rl" + {te = p;p--;{ token( RE_SqOpen ); {stack[top++] = 146; goto st137;} }} + goto st146; +tr369: +#line 1013 "rlscan.rl" + {te = p+1;{ token( RE_SqOpenNeg ); {stack[top++] = 146; goto st137;} }} + goto st146; +tr370: +#line 1002 "rlscan.rl" + {te = p;p--;{ token( TK_Word, ts, te ); }} + goto st146; +tr461: +#line 1081 "rlscan.rl" + {te = p+1;{ token( TK_BarStar ); }} + goto st146; +st146: +#line 1 "NONE" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof146; +case 146: +#line 1 "NONE" + {ts = p;} +#line 4470 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr278; + case 9: goto st147; + case 10: goto tr280; + case 13: goto st147; + case 32: goto st147; + case 34: goto tr281; + case 35: goto tr282; + case 36: goto st151; + case 37: goto st152; + case 39: goto tr285; + case 42: goto st154; + case 45: goto st155; + case 46: goto st156; + case 47: goto tr289; + case 48: goto tr290; + case 58: goto st160; + case 60: goto st162; + case 61: goto st164; + case 62: goto st165; + case 64: goto st166; + case 91: goto st168; + case 95: goto tr297; + case 97: goto st169; + case 101: goto st183; + case 102: goto st190; + case 103: goto st193; + case 105: goto st198; + case 108: goto st211; + case 109: goto st214; + case 111: goto st220; + case 112: goto st226; + case 116: goto st237; + case 118: goto st238; + case 119: goto st245; + case 123: goto tr311; + case 124: goto st251; + case 125: goto tr313; + } + if ( (*p) < 65 ) { + if ( 49 <= (*p) && (*p) <= 57 ) + goto st158; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr277; +st147: + if ( ++p == pe ) + goto _test_eof147; +case 147: + switch( (*p) ) { + case 9: goto st147; + case 13: goto st147; + case 32: goto st147; + } + goto tr314; +tr281: +#line 1 "NONE" + {te = p+1;} + goto st148; +st148: + if ( ++p == pe ) + goto _test_eof148; +case 148: +#line 4537 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr47; + case 34: goto st149; + case 92: goto st25; + } + goto st24; +tr47: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st24; +st24: + if ( ++p == pe ) + goto _test_eof24; +case 24: +#line 4556 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr47; + case 34: goto st149; + case 92: goto st25; + } + goto st24; +st149: + if ( ++p == pe ) + goto _test_eof149; +case 149: + if ( (*p) == 105 ) + goto tr317; + goto tr316; +st25: + if ( ++p == pe ) + goto _test_eof25; +case 25: + if ( (*p) == 10 ) + goto tr47; + goto st24; +tr282: +#line 1 "NONE" + {te = p+1;} + goto st150; +st150: + if ( ++p == pe ) + goto _test_eof150; +case 150: +#line 4585 "rlscan.cpp" + if ( (*p) == 10 ) + goto tr51; + goto st26; +st26: + if ( ++p == pe ) + goto _test_eof26; +case 26: + if ( (*p) == 10 ) + goto tr51; + goto st26; +st151: + if ( ++p == pe ) + goto _test_eof151; +case 151: + switch( (*p) ) { + case 33: goto tr318; + case 42: goto tr319; + case 47: goto tr320; + case 63: goto tr321; + case 94: goto tr322; + case 126: goto tr323; + } + goto tr315; +st152: + if ( ++p == pe ) + goto _test_eof152; +case 152: + switch( (*p) ) { + case 33: goto tr324; + case 42: goto tr325; + case 47: goto tr326; + case 63: goto tr327; + case 94: goto tr328; + case 126: goto tr329; + } + goto tr315; +tr285: +#line 1 "NONE" + {te = p+1;} + goto st153; +st153: + if ( ++p == pe ) + goto _test_eof153; +case 153: +#line 4630 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr53; + case 39: goto st149; + case 92: goto st28; + } + goto st27; +tr53: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st27; +st27: + if ( ++p == pe ) + goto _test_eof27; +case 27: +#line 4649 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr53; + case 39: goto st149; + case 92: goto st28; + } + goto st27; +st28: + if ( ++p == pe ) + goto _test_eof28; +case 28: + if ( (*p) == 10 ) + goto tr53; + goto st27; +st154: + if ( ++p == pe ) + goto _test_eof154; +case 154: + if ( (*p) == 42 ) + goto tr330; + goto tr315; +st155: + if ( ++p == pe ) + goto _test_eof155; +case 155: + switch( (*p) ) { + case 45: goto tr331; + case 62: goto tr332; + } + goto tr315; +st156: + if ( ++p == pe ) + goto _test_eof156; +case 156: + if ( (*p) == 46 ) + goto tr333; + goto tr315; +tr290: +#line 1 "NONE" + {te = p+1;} + goto st157; +st157: + if ( ++p == pe ) + goto _test_eof157; +case 157: +#line 4694 "rlscan.cpp" + if ( (*p) == 120 ) + goto st29; + if ( 48 <= (*p) && (*p) <= 57 ) + goto st158; + goto tr334; +st158: + if ( ++p == pe ) + goto _test_eof158; +case 158: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st158; + goto tr334; +st29: + if ( ++p == pe ) + goto _test_eof29; +case 29: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st159; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st159; + } else + goto st159; + goto tr55; +st159: + if ( ++p == pe ) + goto _test_eof159; +case 159: + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st159; + } else if ( (*p) > 70 ) { + if ( 97 <= (*p) && (*p) <= 102 ) + goto st159; + } else + goto st159; + goto tr336; +st160: + if ( ++p == pe ) + goto _test_eof160; +case 160: + switch( (*p) ) { + case 58: goto tr337; + case 61: goto tr338; + case 62: goto st161; + } + goto tr315; +st161: + if ( ++p == pe ) + goto _test_eof161; +case 161: + if ( (*p) == 62 ) + goto tr341; + goto tr340; +st162: + if ( ++p == pe ) + goto _test_eof162; +case 162: + switch( (*p) ) { + case 33: goto tr342; + case 42: goto tr343; + case 47: goto tr344; + case 58: goto tr345; + case 62: goto st163; + case 94: goto tr347; + case 126: goto tr348; + } + goto tr315; +st163: + if ( ++p == pe ) + goto _test_eof163; +case 163: + switch( (*p) ) { + case 33: goto tr350; + case 42: goto tr351; + case 47: goto tr352; + case 94: goto tr353; + case 126: goto tr354; + } + goto tr349; +st164: + if ( ++p == pe ) + goto _test_eof164; +case 164: + if ( (*p) == 62 ) + goto tr355; + goto tr315; +st165: + if ( ++p == pe ) + goto _test_eof165; +case 165: + switch( (*p) ) { + case 33: goto tr356; + case 42: goto tr357; + case 47: goto tr358; + case 63: goto tr359; + case 94: goto tr360; + case 126: goto tr361; + } + goto tr315; +st166: + if ( ++p == pe ) + goto _test_eof166; +case 166: + switch( (*p) ) { + case 33: goto tr362; + case 42: goto tr363; + case 47: goto tr364; + case 94: goto tr365; + case 126: goto tr366; + } + goto tr315; +tr297: +#line 1 "NONE" + {te = p+1;} +#line 1002 "rlscan.rl" + {act = 108;} + goto st167; +tr377: +#line 1 "NONE" + {te = p+1;} +#line 975 "rlscan.rl" + {act = 97;} + goto st167; +tr380: +#line 1 "NONE" + {te = p+1;} +#line 959 "rlscan.rl" + {act = 92;} + goto st167; +tr386: +#line 1 "NONE" + {te = p+1;} +#line 960 "rlscan.rl" + {act = 93;} + goto st167; +tr390: +#line 1 "NONE" + {te = p+1;} +#line 994 "rlscan.rl" + {act = 102;} + goto st167; +tr391: +#line 1 "NONE" + {te = p+1;} +#line 995 "rlscan.rl" + {act = 103;} + goto st167; +tr395: +#line 1 "NONE" + {te = p+1;} +#line 999 "rlscan.rl" + {act = 107;} + goto st167; +tr398: +#line 1 "NONE" + {te = p+1;} +#line 998 "rlscan.rl" + {act = 106;} + goto st167; +tr403: +#line 1 "NONE" + {te = p+1;} +#line 967 "rlscan.rl" + {act = 96;} + goto st167; +tr409: +#line 1 "NONE" + {te = p+1;} +#line 954 "rlscan.rl" + {act = 90;} + goto st167; +tr415: +#line 1 "NONE" + {te = p+1;} +#line 953 "rlscan.rl" + {act = 89;} + goto st167; +tr418: +#line 1 "NONE" + {te = p+1;} +#line 992 "rlscan.rl" + {act = 100;} + goto st167; +tr421: +#line 1 "NONE" + {te = p+1;} +#line 996 "rlscan.rl" + {act = 104;} + goto st167; +tr427: +#line 1 "NONE" + {te = p+1;} +#line 952 "rlscan.rl" + {act = 88;} + goto st167; +tr433: +#line 1 "NONE" + {te = p+1;} +#line 993 "rlscan.rl" + {act = 101;} + goto st167; +tr440: +#line 1 "NONE" + {te = p+1;} +#line 962 "rlscan.rl" + {act = 95;} + goto st167; +tr445: +#line 1 "NONE" + {te = p+1;} +#line 961 "rlscan.rl" + {act = 94;} + goto st167; +tr446: +#line 1 "NONE" + {te = p+1;} +#line 997 "rlscan.rl" + {act = 105;} + goto st167; +tr453: +#line 1 "NONE" + {te = p+1;} +#line 983 "rlscan.rl" + {act = 98;} + goto st167; +tr457: +#line 1 "NONE" + {te = p+1;} +#line 991 "rlscan.rl" + {act = 99;} + goto st167; +tr460: +#line 1 "NONE" + {te = p+1;} +#line 955 "rlscan.rl" + {act = 91;} + goto st167; +st167: + if ( ++p == pe ) + goto _test_eof167; +case 167: +#line 4938 "rlscan.cpp" + if ( (*p) == 95 ) + goto tr297; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr367; +st168: + if ( ++p == pe ) + goto _test_eof168; +case 168: + if ( (*p) == 94 ) + goto tr369; + goto tr368; +st169: + if ( ++p == pe ) + goto _test_eof169; +case 169: + switch( (*p) ) { + case 95: goto tr297; + case 99: goto st170; + case 108: goto st177; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st170: + if ( ++p == pe ) + goto _test_eof170; +case 170: + switch( (*p) ) { + case 95: goto tr297; + case 99: goto st171; + case 116: goto st174; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st171: + if ( ++p == pe ) + goto _test_eof171; +case 171: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st172; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st172: + if ( ++p == pe ) + goto _test_eof172; +case 172: + switch( (*p) ) { + case 95: goto tr297; + case 115: goto st173; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st173: + if ( ++p == pe ) + goto _test_eof173; +case 173: + switch( (*p) ) { + case 95: goto tr297; + case 115: goto tr377; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st174: + if ( ++p == pe ) + goto _test_eof174; +case 174: + switch( (*p) ) { + case 95: goto tr297; + case 105: goto st175; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st175: + if ( ++p == pe ) + goto _test_eof175; +case 175: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st176; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st176: + if ( ++p == pe ) + goto _test_eof176; +case 176: + switch( (*p) ) { + case 95: goto tr297; + case 110: goto tr380; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st177: + if ( ++p == pe ) + goto _test_eof177; +case 177: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st178; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st178: + if ( ++p == pe ) + goto _test_eof178; +case 178: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto st179; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st179: + if ( ++p == pe ) + goto _test_eof179; +case 179: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto st180; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st180: + if ( ++p == pe ) + goto _test_eof180; +case 180: + switch( (*p) ) { + case 95: goto tr297; + case 121: goto st181; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st181: + if ( ++p == pe ) + goto _test_eof181; +case 181: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st182; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st182: + if ( ++p == pe ) + goto _test_eof182; +case 182: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto tr386; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st183: + if ( ++p == pe ) + goto _test_eof183; +case 183: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st184; + case 114: goto st185; + case 120: goto st186; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st184: + if ( ++p == pe ) + goto _test_eof184; +case 184: + switch( (*p) ) { + case 95: goto tr297; + case 102: goto tr390; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st185: + if ( ++p == pe ) + goto _test_eof185; +case 185: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto tr391; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st186: + if ( ++p == pe ) + goto _test_eof186; +case 186: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st187; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st187: + if ( ++p == pe ) + goto _test_eof187; +case 187: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st188; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st188: + if ( ++p == pe ) + goto _test_eof188; +case 188: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto st189; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st189: + if ( ++p == pe ) + goto _test_eof189; +case 189: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto tr395; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st190: + if ( ++p == pe ) + goto _test_eof190; +case 190: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto st191; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st191: + if ( ++p == pe ) + goto _test_eof191; +case 191: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st192; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st192: + if ( ++p == pe ) + goto _test_eof192; +case 192: + switch( (*p) ) { + case 95: goto tr297; + case 109: goto tr398; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st193: + if ( ++p == pe ) + goto _test_eof193; +case 193: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st194; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st194: + if ( ++p == pe ) + goto _test_eof194; +case 194: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto st195; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st195: + if ( ++p == pe ) + goto _test_eof195; +case 195: + switch( (*p) ) { + case 95: goto tr297; + case 107: goto st196; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st196: + if ( ++p == pe ) + goto _test_eof196; +case 196: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st197; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st197: + if ( ++p == pe ) + goto _test_eof197; +case 197: + switch( (*p) ) { + case 95: goto tr297; + case 121: goto tr403; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st198: + if ( ++p == pe ) + goto _test_eof198; +case 198: + switch( (*p) ) { + case 95: goto tr297; + case 109: goto st199; + case 110: goto st203; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st199: + if ( ++p == pe ) + goto _test_eof199; +case 199: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st200; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st200: + if ( ++p == pe ) + goto _test_eof200; +case 200: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st201; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st201: + if ( ++p == pe ) + goto _test_eof201; +case 201: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto st202; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st202: + if ( ++p == pe ) + goto _test_eof202; +case 202: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto tr409; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st203: + if ( ++p == pe ) + goto _test_eof203; +case 203: + switch( (*p) ) { + case 95: goto tr297; + case 99: goto st204; + case 119: goto st208; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st204: + if ( ++p == pe ) + goto _test_eof204; +case 204: + switch( (*p) ) { + case 95: goto tr297; + case 108: goto st205; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st205: + if ( ++p == pe ) + goto _test_eof205; +case 205: + switch( (*p) ) { + case 95: goto tr297; + case 117: goto st206; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st206: + if ( ++p == pe ) + goto _test_eof206; +case 206: + switch( (*p) ) { + case 95: goto tr297; + case 100: goto st207; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st207: + if ( ++p == pe ) + goto _test_eof207; +case 207: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto tr415; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st208: + if ( ++p == pe ) + goto _test_eof208; +case 208: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto st209; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st209: + if ( ++p == pe ) + goto _test_eof209; +case 209: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st210; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st210: + if ( ++p == pe ) + goto _test_eof210; +case 210: + switch( (*p) ) { + case 95: goto tr297; + case 110: goto tr418; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st211: + if ( ++p == pe ) + goto _test_eof211; +case 211: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st212; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st212: + if ( ++p == pe ) + goto _test_eof212; +case 212: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto st213; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st213: + if ( ++p == pe ) + goto _test_eof213; +case 213: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto tr421; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st214: + if ( ++p == pe ) + goto _test_eof214; +case 214: + switch( (*p) ) { + case 95: goto tr297; + case 97: goto st215; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st215: + if ( ++p == pe ) + goto _test_eof215; +case 215: + switch( (*p) ) { + case 95: goto tr297; + case 99: goto st216; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st216: + if ( ++p == pe ) + goto _test_eof216; +case 216: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto st217; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st217: + if ( ++p == pe ) + goto _test_eof217; +case 217: + switch( (*p) ) { + case 95: goto tr297; + case 105: goto st218; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st218: + if ( ++p == pe ) + goto _test_eof218; +case 218: + switch( (*p) ) { + case 95: goto tr297; + case 110: goto st219; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st219: + if ( ++p == pe ) + goto _test_eof219; +case 219: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto tr427; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st220: + if ( ++p == pe ) + goto _test_eof220; +case 220: + switch( (*p) ) { + case 95: goto tr297; + case 117: goto st221; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st221: + if ( ++p == pe ) + goto _test_eof221; +case 221: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto st222; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st222: + if ( ++p == pe ) + goto _test_eof222; +case 222: + switch( (*p) ) { + case 95: goto tr297; + case 119: goto st223; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st223: + if ( ++p == pe ) + goto _test_eof223; +case 223: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto st224; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st224: + if ( ++p == pe ) + goto _test_eof224; +case 224: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st225; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st225: + if ( ++p == pe ) + goto _test_eof225; +case 225: + switch( (*p) ) { + case 95: goto tr297; + case 110: goto tr433; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st226: + if ( ++p == pe ) + goto _test_eof226; +case 226: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st227; + case 114: goto st232; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st227: + if ( ++p == pe ) + goto _test_eof227; +case 227: + switch( (*p) ) { + case 95: goto tr297; + case 115: goto st228; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st228: + if ( ++p == pe ) + goto _test_eof228; +case 228: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto st229; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st229: + if ( ++p == pe ) + goto _test_eof229; +case 229: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st230; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st230: + if ( ++p == pe ) + goto _test_eof230; +case 230: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto st231; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st231: + if ( ++p == pe ) + goto _test_eof231; +case 231: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto tr440; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st232: + if ( ++p == pe ) + goto _test_eof232; +case 232: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st233; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st233: + if ( ++p == pe ) + goto _test_eof233; +case 233: + switch( (*p) ) { + case 95: goto tr297; + case 112: goto st234; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st234: + if ( ++p == pe ) + goto _test_eof234; +case 234: + switch( (*p) ) { + case 95: goto tr297; + case 117: goto st235; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st235: + if ( ++p == pe ) + goto _test_eof235; +case 235: + switch( (*p) ) { + case 95: goto tr297; + case 115: goto st236; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st236: + if ( ++p == pe ) + goto _test_eof236; +case 236: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto tr445; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st237: + if ( ++p == pe ) + goto _test_eof237; +case 237: + switch( (*p) ) { + case 95: goto tr297; + case 111: goto tr446; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st238: + if ( ++p == pe ) + goto _test_eof238; +case 238: + switch( (*p) ) { + case 95: goto tr297; + case 97: goto st239; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st239: + if ( ++p == pe ) + goto _test_eof239; +case 239: + switch( (*p) ) { + case 95: goto tr297; + case 114: goto st240; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st240: + if ( ++p == pe ) + goto _test_eof240; +case 240: + switch( (*p) ) { + case 95: goto tr297; + case 105: goto st241; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st241: + if ( ++p == pe ) + goto _test_eof241; +case 241: + switch( (*p) ) { + case 95: goto tr297; + case 97: goto st242; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 98 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st242: + if ( ++p == pe ) + goto _test_eof242; +case 242: + switch( (*p) ) { + case 95: goto tr297; + case 98: goto st243; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st243: + if ( ++p == pe ) + goto _test_eof243; +case 243: + switch( (*p) ) { + case 95: goto tr297; + case 108: goto st244; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st244: + if ( ++p == pe ) + goto _test_eof244; +case 244: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto tr453; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st245: + if ( ++p == pe ) + goto _test_eof245; +case 245: + switch( (*p) ) { + case 95: goto tr297; + case 104: goto st246; + case 114: goto st248; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st246: + if ( ++p == pe ) + goto _test_eof246; +case 246: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto st247; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st247: + if ( ++p == pe ) + goto _test_eof247; +case 247: + switch( (*p) ) { + case 95: goto tr297; + case 110: goto tr457; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st248: + if ( ++p == pe ) + goto _test_eof248; +case 248: + switch( (*p) ) { + case 95: goto tr297; + case 105: goto st249; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st249: + if ( ++p == pe ) + goto _test_eof249; +case 249: + switch( (*p) ) { + case 95: goto tr297; + case 116: goto st250; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st250: + if ( ++p == pe ) + goto _test_eof250; +case 250: + switch( (*p) ) { + case 95: goto tr297; + case 101: goto tr460; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto tr297; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto tr297; + } else + goto tr297; + goto tr370; +st251: + if ( ++p == pe ) + goto _test_eof251; +case 251: + if ( (*p) == 42 ) + goto tr461; + goto tr315; +tr313: +#line 1 "NONE" + {te = p+1;} + goto st252; +st252: + if ( ++p == pe ) + goto _test_eof252; +case 252: +#line 6374 "rlscan.cpp" + if ( (*p) == 37 ) + goto st30; + goto tr315; +st30: + if ( ++p == pe ) + goto _test_eof30; +case 30: + if ( (*p) == 37 ) + goto tr57; + goto tr45; +tr58: +#line 1146 "rlscan.rl" + {{p = ((te))-1;}{ pass( *ts, 0, 0 ); }} + goto st253; +tr61: +#line 1130 "rlscan.rl" + {te = p+1;{ pass( IMP_Literal, ts, te ); }} + goto st253; +tr64: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } +#line 1128 "rlscan.rl" + {te = p+1;{ pass(); }} + goto st253; +tr463: +#line 1146 "rlscan.rl" + {te = p+1;{ pass( *ts, 0, 0 ); }} + goto st253; +tr464: +#line 1145 "rlscan.rl" + {te = p+1;} + goto st253; +tr474: +#line 1144 "rlscan.rl" + {te = p;p--;{ pass(); }} + goto st253; +tr475: +#line 1146 "rlscan.rl" + {te = p;p--;{ pass( *ts, 0, 0 ); }} + goto st253; +tr477: +#line 1138 "rlscan.rl" + {te = p;p--;{ + updateCol(); + singleLineSpec = true; + startSection(); + {stack[top++] = 253; goto st146;} + }} + goto st253; +tr478: +#line 1132 "rlscan.rl" + {te = p+1;{ + updateCol(); + singleLineSpec = false; + startSection(); + {stack[top++] = 253; goto st146;} + }} + goto st253; +tr479: +#line 1127 "rlscan.rl" + {te = p;p--;{ pass( IMP_UInt, ts, te ); }} + goto st253; +tr480: +#line 1126 "rlscan.rl" + {te = p;p--;{ pass( IMP_Word, ts, te ); }} + goto st253; +st253: +#line 1 "NONE" + {ts = 0;} + if ( ++p == pe ) + goto _test_eof253; +case 253: +#line 1 "NONE" + {ts = p;} +#line 6453 "rlscan.cpp" + switch( (*p) ) { + case 0: goto tr464; + case 9: goto st254; + case 10: goto tr466; + case 32: goto st254; + case 34: goto tr467; + case 35: goto tr468; + case 37: goto st257; + case 39: goto tr470; + case 47: goto tr471; + case 95: goto st262; + } + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st261; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto st262; + } else + goto st262; + goto tr463; +tr466: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st254; +st254: + if ( ++p == pe ) + goto _test_eof254; +case 254: +#line 6487 "rlscan.cpp" + switch( (*p) ) { + case 9: goto st254; + case 10: goto tr466; + case 32: goto st254; + } + goto tr474; +tr467: +#line 1 "NONE" + {te = p+1;} + goto st255; +st255: + if ( ++p == pe ) + goto _test_eof255; +case 255: +#line 6502 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr60; + case 34: goto tr61; + case 92: goto st32; + } + goto st31; +tr60: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st31; +st31: + if ( ++p == pe ) + goto _test_eof31; +case 31: +#line 6521 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr60; + case 34: goto tr61; + case 92: goto st32; + } + goto st31; +st32: + if ( ++p == pe ) + goto _test_eof32; +case 32: + if ( (*p) == 10 ) + goto tr60; + goto st31; +tr468: +#line 1 "NONE" + {te = p+1;} + goto st256; +st256: + if ( ++p == pe ) + goto _test_eof256; +case 256: +#line 6543 "rlscan.cpp" + if ( (*p) == 10 ) + goto tr64; + goto st33; +st33: + if ( ++p == pe ) + goto _test_eof33; +case 33: + if ( (*p) == 10 ) + goto tr64; + goto st33; +st257: + if ( ++p == pe ) + goto _test_eof257; +case 257: + if ( (*p) == 37 ) + goto st258; + goto tr475; +st258: + if ( ++p == pe ) + goto _test_eof258; +case 258: + if ( (*p) == 123 ) + goto tr478; + goto tr477; +tr470: +#line 1 "NONE" + {te = p+1;} + goto st259; +st259: + if ( ++p == pe ) + goto _test_eof259; +case 259: +#line 6576 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr66; + case 39: goto tr61; + case 92: goto st35; + } + goto st34; +tr66: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st34; +st34: + if ( ++p == pe ) + goto _test_eof34; +case 34: +#line 6595 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr66; + case 39: goto tr61; + case 92: goto st35; + } + goto st34; +st35: + if ( ++p == pe ) + goto _test_eof35; +case 35: + if ( (*p) == 10 ) + goto tr66; + goto st34; +tr471: +#line 1 "NONE" + {te = p+1;} + goto st260; +st260: + if ( ++p == pe ) + goto _test_eof260; +case 260: +#line 6617 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr69; + case 47: goto tr61; + case 92: goto st37; + } + goto st36; +tr69: +#line 641 "rlscan.rl" + { + lastnl = p; + column = 0; + line++; + } + goto st36; +st36: + if ( ++p == pe ) + goto _test_eof36; +case 36: +#line 6636 "rlscan.cpp" + switch( (*p) ) { + case 10: goto tr69; + case 47: goto tr61; + case 92: goto st37; + } + goto st36; +st37: + if ( ++p == pe ) + goto _test_eof37; +case 37: + if ( (*p) == 10 ) + goto tr69; + goto st36; +st261: + if ( ++p == pe ) + goto _test_eof261; +case 261: + if ( 48 <= (*p) && (*p) <= 57 ) + goto st261; + goto tr479; +st262: + if ( ++p == pe ) + goto _test_eof262; +case 262: + if ( (*p) == 95 ) + goto st262; + if ( (*p) < 65 ) { + if ( 48 <= (*p) && (*p) <= 57 ) + goto st262; + } else if ( (*p) > 90 ) { + if ( 97 <= (*p) && (*p) <= 122 ) + goto st262; + } else + goto st262; + goto tr480; + } + _test_eof38: cs = 38; goto _test_eof; + _test_eof39: cs = 39; goto _test_eof; + _test_eof40: cs = 40; goto _test_eof; + _test_eof1: cs = 1; goto _test_eof; + _test_eof2: cs = 2; goto _test_eof; + _test_eof41: cs = 41; goto _test_eof; + _test_eof42: cs = 42; goto _test_eof; + _test_eof43: cs = 43; goto _test_eof; + _test_eof3: cs = 3; goto _test_eof; + _test_eof4: cs = 4; goto _test_eof; + _test_eof44: cs = 44; goto _test_eof; + _test_eof5: cs = 5; goto _test_eof; + _test_eof6: cs = 6; goto _test_eof; + _test_eof7: cs = 7; goto _test_eof; + _test_eof45: cs = 45; goto _test_eof; + _test_eof46: cs = 46; goto _test_eof; + _test_eof47: cs = 47; goto _test_eof; + _test_eof48: cs = 48; goto _test_eof; + _test_eof49: cs = 49; goto _test_eof; + _test_eof50: cs = 50; goto _test_eof; + _test_eof51: cs = 51; goto _test_eof; + _test_eof52: cs = 52; goto _test_eof; + _test_eof53: cs = 53; goto _test_eof; + _test_eof54: cs = 54; goto _test_eof; + _test_eof8: cs = 8; goto _test_eof; + _test_eof9: cs = 9; goto _test_eof; + _test_eof55: cs = 55; goto _test_eof; + _test_eof10: cs = 10; goto _test_eof; + _test_eof56: cs = 56; goto _test_eof; + _test_eof11: cs = 11; goto _test_eof; + _test_eof12: cs = 12; goto _test_eof; + _test_eof57: cs = 57; goto _test_eof; + _test_eof13: cs = 13; goto _test_eof; + _test_eof14: cs = 14; goto _test_eof; + _test_eof58: cs = 58; goto _test_eof; + _test_eof59: cs = 59; goto _test_eof; + _test_eof15: cs = 15; goto _test_eof; + _test_eof60: cs = 60; goto _test_eof; + _test_eof61: cs = 61; goto _test_eof; + _test_eof62: cs = 62; goto _test_eof; + _test_eof63: cs = 63; goto _test_eof; + _test_eof64: cs = 64; goto _test_eof; + _test_eof65: cs = 65; goto _test_eof; + _test_eof66: cs = 66; goto _test_eof; + _test_eof67: cs = 67; goto _test_eof; + _test_eof68: cs = 68; goto _test_eof; + _test_eof69: cs = 69; goto _test_eof; + _test_eof70: cs = 70; goto _test_eof; + _test_eof71: cs = 71; goto _test_eof; + _test_eof72: cs = 72; goto _test_eof; + _test_eof73: cs = 73; goto _test_eof; + _test_eof74: cs = 74; goto _test_eof; + _test_eof75: cs = 75; goto _test_eof; + _test_eof76: cs = 76; goto _test_eof; + _test_eof77: cs = 77; goto _test_eof; + _test_eof78: cs = 78; goto _test_eof; + _test_eof79: cs = 79; goto _test_eof; + _test_eof80: cs = 80; goto _test_eof; + _test_eof81: cs = 81; goto _test_eof; + _test_eof82: cs = 82; goto _test_eof; + _test_eof83: cs = 83; goto _test_eof; + _test_eof84: cs = 84; goto _test_eof; + _test_eof85: cs = 85; goto _test_eof; + _test_eof86: cs = 86; goto _test_eof; + _test_eof87: cs = 87; goto _test_eof; + _test_eof88: cs = 88; goto _test_eof; + _test_eof89: cs = 89; goto _test_eof; + _test_eof90: cs = 90; goto _test_eof; + _test_eof91: cs = 91; goto _test_eof; + _test_eof92: cs = 92; goto _test_eof; + _test_eof93: cs = 93; goto _test_eof; + _test_eof94: cs = 94; goto _test_eof; + _test_eof95: cs = 95; goto _test_eof; + _test_eof96: cs = 96; goto _test_eof; + _test_eof97: cs = 97; goto _test_eof; + _test_eof16: cs = 16; goto _test_eof; + _test_eof17: cs = 17; goto _test_eof; + _test_eof98: cs = 98; goto _test_eof; + _test_eof18: cs = 18; goto _test_eof; + _test_eof19: cs = 19; goto _test_eof; + _test_eof99: cs = 99; goto _test_eof; + _test_eof20: cs = 20; goto _test_eof; + _test_eof21: cs = 21; goto _test_eof; + _test_eof22: cs = 22; goto _test_eof; + _test_eof100: cs = 100; goto _test_eof; + _test_eof101: cs = 101; goto _test_eof; + _test_eof23: cs = 23; goto _test_eof; + _test_eof102: cs = 102; goto _test_eof; + _test_eof103: cs = 103; goto _test_eof; + _test_eof104: cs = 104; goto _test_eof; + _test_eof105: cs = 105; goto _test_eof; + _test_eof106: cs = 106; goto _test_eof; + _test_eof107: cs = 107; goto _test_eof; + _test_eof108: cs = 108; goto _test_eof; + _test_eof109: cs = 109; goto _test_eof; + _test_eof110: cs = 110; goto _test_eof; + _test_eof111: cs = 111; goto _test_eof; + _test_eof112: cs = 112; goto _test_eof; + _test_eof113: cs = 113; goto _test_eof; + _test_eof114: cs = 114; goto _test_eof; + _test_eof115: cs = 115; goto _test_eof; + _test_eof116: cs = 116; goto _test_eof; + _test_eof117: cs = 117; goto _test_eof; + _test_eof118: cs = 118; goto _test_eof; + _test_eof119: cs = 119; goto _test_eof; + _test_eof120: cs = 120; goto _test_eof; + _test_eof121: cs = 121; goto _test_eof; + _test_eof122: cs = 122; goto _test_eof; + _test_eof123: cs = 123; goto _test_eof; + _test_eof124: cs = 124; goto _test_eof; + _test_eof125: cs = 125; goto _test_eof; + _test_eof126: cs = 126; goto _test_eof; + _test_eof127: cs = 127; goto _test_eof; + _test_eof128: cs = 128; goto _test_eof; + _test_eof129: cs = 129; goto _test_eof; + _test_eof130: cs = 130; goto _test_eof; + _test_eof131: cs = 131; goto _test_eof; + _test_eof132: cs = 132; goto _test_eof; + _test_eof133: cs = 133; goto _test_eof; + _test_eof134: cs = 134; goto _test_eof; + _test_eof135: cs = 135; goto _test_eof; + _test_eof136: cs = 136; goto _test_eof; + _test_eof137: cs = 137; goto _test_eof; + _test_eof138: cs = 138; goto _test_eof; + _test_eof139: cs = 139; goto _test_eof; + _test_eof140: cs = 140; goto _test_eof; + _test_eof141: cs = 141; goto _test_eof; + _test_eof142: cs = 142; goto _test_eof; + _test_eof143: cs = 143; goto _test_eof; + _test_eof144: cs = 144; goto _test_eof; + _test_eof145: cs = 145; goto _test_eof; + _test_eof146: cs = 146; goto _test_eof; + _test_eof147: cs = 147; goto _test_eof; + _test_eof148: cs = 148; goto _test_eof; + _test_eof24: cs = 24; goto _test_eof; + _test_eof149: cs = 149; goto _test_eof; + _test_eof25: cs = 25; goto _test_eof; + _test_eof150: cs = 150; goto _test_eof; + _test_eof26: cs = 26; goto _test_eof; + _test_eof151: cs = 151; goto _test_eof; + _test_eof152: cs = 152; goto _test_eof; + _test_eof153: cs = 153; goto _test_eof; + _test_eof27: cs = 27; goto _test_eof; + _test_eof28: cs = 28; goto _test_eof; + _test_eof154: cs = 154; goto _test_eof; + _test_eof155: cs = 155; goto _test_eof; + _test_eof156: cs = 156; goto _test_eof; + _test_eof157: cs = 157; goto _test_eof; + _test_eof158: cs = 158; goto _test_eof; + _test_eof29: cs = 29; goto _test_eof; + _test_eof159: cs = 159; goto _test_eof; + _test_eof160: cs = 160; goto _test_eof; + _test_eof161: cs = 161; goto _test_eof; + _test_eof162: cs = 162; goto _test_eof; + _test_eof163: cs = 163; goto _test_eof; + _test_eof164: cs = 164; goto _test_eof; + _test_eof165: cs = 165; goto _test_eof; + _test_eof166: cs = 166; goto _test_eof; + _test_eof167: cs = 167; goto _test_eof; + _test_eof168: cs = 168; goto _test_eof; + _test_eof169: cs = 169; goto _test_eof; + _test_eof170: cs = 170; goto _test_eof; + _test_eof171: cs = 171; goto _test_eof; + _test_eof172: cs = 172; goto _test_eof; + _test_eof173: cs = 173; goto _test_eof; + _test_eof174: cs = 174; goto _test_eof; + _test_eof175: cs = 175; goto _test_eof; + _test_eof176: cs = 176; goto _test_eof; + _test_eof177: cs = 177; goto _test_eof; + _test_eof178: cs = 178; goto _test_eof; + _test_eof179: cs = 179; goto _test_eof; + _test_eof180: cs = 180; goto _test_eof; + _test_eof181: cs = 181; goto _test_eof; + _test_eof182: cs = 182; goto _test_eof; + _test_eof183: cs = 183; goto _test_eof; + _test_eof184: cs = 184; goto _test_eof; + _test_eof185: cs = 185; goto _test_eof; + _test_eof186: cs = 186; goto _test_eof; + _test_eof187: cs = 187; goto _test_eof; + _test_eof188: cs = 188; goto _test_eof; + _test_eof189: cs = 189; goto _test_eof; + _test_eof190: cs = 190; goto _test_eof; + _test_eof191: cs = 191; goto _test_eof; + _test_eof192: cs = 192; goto _test_eof; + _test_eof193: cs = 193; goto _test_eof; + _test_eof194: cs = 194; goto _test_eof; + _test_eof195: cs = 195; goto _test_eof; + _test_eof196: cs = 196; goto _test_eof; + _test_eof197: cs = 197; goto _test_eof; + _test_eof198: cs = 198; goto _test_eof; + _test_eof199: cs = 199; goto _test_eof; + _test_eof200: cs = 200; goto _test_eof; + _test_eof201: cs = 201; goto _test_eof; + _test_eof202: cs = 202; goto _test_eof; + _test_eof203: cs = 203; goto _test_eof; + _test_eof204: cs = 204; goto _test_eof; + _test_eof205: cs = 205; goto _test_eof; + _test_eof206: cs = 206; goto _test_eof; + _test_eof207: cs = 207; goto _test_eof; + _test_eof208: cs = 208; goto _test_eof; + _test_eof209: cs = 209; goto _test_eof; + _test_eof210: cs = 210; goto _test_eof; + _test_eof211: cs = 211; goto _test_eof; + _test_eof212: cs = 212; goto _test_eof; + _test_eof213: cs = 213; goto _test_eof; + _test_eof214: cs = 214; goto _test_eof; + _test_eof215: cs = 215; goto _test_eof; + _test_eof216: cs = 216; goto _test_eof; + _test_eof217: cs = 217; goto _test_eof; + _test_eof218: cs = 218; goto _test_eof; + _test_eof219: cs = 219; goto _test_eof; + _test_eof220: cs = 220; goto _test_eof; + _test_eof221: cs = 221; goto _test_eof; + _test_eof222: cs = 222; goto _test_eof; + _test_eof223: cs = 223; goto _test_eof; + _test_eof224: cs = 224; goto _test_eof; + _test_eof225: cs = 225; goto _test_eof; + _test_eof226: cs = 226; goto _test_eof; + _test_eof227: cs = 227; goto _test_eof; + _test_eof228: cs = 228; goto _test_eof; + _test_eof229: cs = 229; goto _test_eof; + _test_eof230: cs = 230; goto _test_eof; + _test_eof231: cs = 231; goto _test_eof; + _test_eof232: cs = 232; goto _test_eof; + _test_eof233: cs = 233; goto _test_eof; + _test_eof234: cs = 234; goto _test_eof; + _test_eof235: cs = 235; goto _test_eof; + _test_eof236: cs = 236; goto _test_eof; + _test_eof237: cs = 237; goto _test_eof; + _test_eof238: cs = 238; goto _test_eof; + _test_eof239: cs = 239; goto _test_eof; + _test_eof240: cs = 240; goto _test_eof; + _test_eof241: cs = 241; goto _test_eof; + _test_eof242: cs = 242; goto _test_eof; + _test_eof243: cs = 243; goto _test_eof; + _test_eof244: cs = 244; goto _test_eof; + _test_eof245: cs = 245; goto _test_eof; + _test_eof246: cs = 246; goto _test_eof; + _test_eof247: cs = 247; goto _test_eof; + _test_eof248: cs = 248; goto _test_eof; + _test_eof249: cs = 249; goto _test_eof; + _test_eof250: cs = 250; goto _test_eof; + _test_eof251: cs = 251; goto _test_eof; + _test_eof252: cs = 252; goto _test_eof; + _test_eof30: cs = 30; goto _test_eof; + _test_eof253: cs = 253; goto _test_eof; + _test_eof254: cs = 254; goto _test_eof; + _test_eof255: cs = 255; goto _test_eof; + _test_eof31: cs = 31; goto _test_eof; + _test_eof32: cs = 32; goto _test_eof; + _test_eof256: cs = 256; goto _test_eof; + _test_eof33: cs = 33; goto _test_eof; + _test_eof257: cs = 257; goto _test_eof; + _test_eof258: cs = 258; goto _test_eof; + _test_eof259: cs = 259; goto _test_eof; + _test_eof34: cs = 34; goto _test_eof; + _test_eof35: cs = 35; goto _test_eof; + _test_eof260: cs = 260; goto _test_eof; + _test_eof36: cs = 36; goto _test_eof; + _test_eof37: cs = 37; goto _test_eof; + _test_eof261: cs = 261; goto _test_eof; + _test_eof262: cs = 262; goto _test_eof; + + _test_eof: {} + if ( p == eof ) + { + switch ( cs ) { + case 39: goto tr82; + case 40: goto tr83; + case 1: goto tr0; + case 2: goto tr0; + case 41: goto tr83; + case 42: goto tr85; + case 43: goto tr83; + case 3: goto tr0; + case 4: goto tr0; + case 44: goto tr83; + case 5: goto tr0; + case 6: goto tr0; + case 7: goto tr0; + case 45: goto tr87; + case 46: goto tr88; + case 47: goto tr89; + case 48: goto tr89; + case 49: goto tr89; + case 50: goto tr89; + case 51: goto tr89; + case 53: goto tr113; + case 54: goto tr114; + case 8: goto tr14; + case 9: goto tr14; + case 55: goto tr114; + case 10: goto tr14; + case 56: goto tr114; + case 11: goto tr14; + case 12: goto tr14; + case 57: goto tr114; + case 13: goto tr14; + case 14: goto tr14; + case 58: goto tr115; + case 59: goto tr115; + case 15: goto tr27; + case 60: goto tr117; + case 61: goto tr114; + case 62: goto tr119; + case 63: goto tr120; + case 64: goto tr120; + case 65: goto tr120; + case 66: goto tr120; + case 67: goto tr120; + case 68: goto tr134; + case 69: goto tr120; + case 70: goto tr120; + case 71: goto tr120; + case 72: goto tr120; + case 73: goto tr120; + case 74: goto tr120; + case 75: goto tr120; + case 76: goto tr120; + case 77: goto tr120; + case 78: goto tr120; + case 79: goto tr120; + case 80: goto tr120; + case 81: goto tr120; + case 82: goto tr120; + case 83: goto tr120; + case 84: goto tr120; + case 85: goto tr120; + case 86: goto tr120; + case 87: goto tr120; + case 88: goto tr120; + case 89: goto tr120; + case 90: goto tr120; + case 91: goto tr120; + case 92: goto tr120; + case 93: goto tr120; + case 94: goto tr120; + case 96: goto tr181; + case 97: goto tr182; + case 16: goto tr29; + case 17: goto tr29; + case 98: goto tr182; + case 18: goto tr29; + case 19: goto tr29; + case 99: goto tr182; + case 20: goto tr29; + case 21: goto tr29; + case 22: goto tr29; + case 100: goto tr183; + case 101: goto tr183; + case 23: goto tr43; + case 102: goto tr185; + case 103: goto tr182; + case 104: goto tr187; + case 105: goto tr188; + case 106: goto tr188; + case 107: goto tr188; + case 108: goto tr188; + case 109: goto tr188; + case 110: goto tr202; + case 111: goto tr188; + case 112: goto tr188; + case 113: goto tr188; + case 114: goto tr188; + case 115: goto tr188; + case 116: goto tr188; + case 117: goto tr188; + case 118: goto tr188; + case 119: goto tr188; + case 120: goto tr188; + case 121: goto tr188; + case 122: goto tr188; + case 123: goto tr188; + case 124: goto tr188; + case 125: goto tr188; + case 126: goto tr188; + case 127: goto tr188; + case 128: goto tr188; + case 129: goto tr188; + case 130: goto tr188; + case 131: goto tr188; + case 132: goto tr188; + case 133: goto tr188; + case 134: goto tr188; + case 135: goto tr188; + case 136: goto tr188; + case 138: goto tr237; + case 140: goto tr255; + case 141: goto tr257; + case 142: goto tr259; + case 144: goto tr275; + case 145: goto tr276; + case 147: goto tr314; + case 148: goto tr315; + case 24: goto tr45; + case 149: goto tr316; + case 25: goto tr45; + case 150: goto tr315; + case 26: goto tr45; + case 151: goto tr315; + case 152: goto tr315; + case 153: goto tr315; + case 27: goto tr45; + case 28: goto tr45; + case 154: goto tr315; + case 155: goto tr315; + case 156: goto tr315; + case 157: goto tr334; + case 158: goto tr334; + case 29: goto tr55; + case 159: goto tr336; + case 160: goto tr315; + case 161: goto tr340; + case 162: goto tr315; + case 163: goto tr349; + case 164: goto tr315; + case 165: goto tr315; + case 166: goto tr315; + case 167: goto tr367; + case 168: goto tr368; + case 169: goto tr370; + case 170: goto tr370; + case 171: goto tr370; + case 172: goto tr370; + case 173: goto tr370; + case 174: goto tr370; + case 175: goto tr370; + case 176: goto tr370; + case 177: goto tr370; + case 178: goto tr370; + case 179: goto tr370; + case 180: goto tr370; + case 181: goto tr370; + case 182: goto tr370; + case 183: goto tr370; + case 184: goto tr370; + case 185: goto tr370; + case 186: goto tr370; + case 187: goto tr370; + case 188: goto tr370; + case 189: goto tr370; + case 190: goto tr370; + case 191: goto tr370; + case 192: goto tr370; + case 193: goto tr370; + case 194: goto tr370; + case 195: goto tr370; + case 196: goto tr370; + case 197: goto tr370; + case 198: goto tr370; + case 199: goto tr370; + case 200: goto tr370; + case 201: goto tr370; + case 202: goto tr370; + case 203: goto tr370; + case 204: goto tr370; + case 205: goto tr370; + case 206: goto tr370; + case 207: goto tr370; + case 208: goto tr370; + case 209: goto tr370; + case 210: goto tr370; + case 211: goto tr370; + case 212: goto tr370; + case 213: goto tr370; + case 214: goto tr370; + case 215: goto tr370; + case 216: goto tr370; + case 217: goto tr370; + case 218: goto tr370; + case 219: goto tr370; + case 220: goto tr370; + case 221: goto tr370; + case 222: goto tr370; + case 223: goto tr370; + case 224: goto tr370; + case 225: goto tr370; + case 226: goto tr370; + case 227: goto tr370; + case 228: goto tr370; + case 229: goto tr370; + case 230: goto tr370; + case 231: goto tr370; + case 232: goto tr370; + case 233: goto tr370; + case 234: goto tr370; + case 235: goto tr370; + case 236: goto tr370; + case 237: goto tr370; + case 238: goto tr370; + case 239: goto tr370; + case 240: goto tr370; + case 241: goto tr370; + case 242: goto tr370; + case 243: goto tr370; + case 244: goto tr370; + case 245: goto tr370; + case 246: goto tr370; + case 247: goto tr370; + case 248: goto tr370; + case 249: goto tr370; + case 250: goto tr370; + case 251: goto tr315; + case 252: goto tr315; + case 30: goto tr45; + case 254: goto tr474; + case 255: goto tr475; + case 31: goto tr58; + case 32: goto tr58; + case 256: goto tr475; + case 33: goto tr58; + case 257: goto tr475; + case 258: goto tr477; + case 259: goto tr475; + case 34: goto tr58; + case 35: goto tr58; + case 260: goto tr475; + case 36: goto tr58; + case 37: goto tr58; + case 261: goto tr479; + case 262: goto tr480; + } + } + + _out: {} + } + +#line 1241 "rlscan.rl" + + /* Check if we failed. */ + if ( cs == rlscan_error ) { + /* Machine failed before finding a token. I'm not yet sure if this + * is reachable. */ + scan_error() << "scanner error" << endl; + exit(1); + } + + /* Decide if we need to preserve anything. */ + char *preserve = ts; + + /* Now set up the prefix. */ + if ( preserve == 0 ) + have = 0; + else { + /* There is data that needs to be shifted over. */ + have = pe - preserve; + memmove( buf, preserve, have ); + unsigned int shiftback = preserve - buf; + if ( ts != 0 ) + ts -= shiftback; + te -= shiftback; + + preserve = buf; + } + } + + delete[] buf; +} diff --git a/contrib/tools/ragel6/rlscan.h b/contrib/tools/ragel6/rlscan.h new file mode 100644 index 0000000000..2aa6ae3914 --- /dev/null +++ b/contrib/tools/ragel6/rlscan.h @@ -0,0 +1,132 @@ +/* + * Copyright 2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RLSCAN_H +#define _RLSCAN_H + +#include <iostream> +#include "rlscan.h" +#include "vector.h" +#include "rlparse.h" +#include "parsedata.h" +#include "avltree.h" +#include "vector.h" + +using std::istream; +using std::ostream; + +extern const char *Parser_lelNames[]; + +struct Scanner +{ + Scanner( InputData &id, const char *fileName, istream &input, + Parser *inclToParser, char *inclSectionTarg, + int includeDepth, bool importMachines ) + : + id(id), fileName(fileName), + input(input), + inclToParser(inclToParser), + inclSectionTarg(inclSectionTarg), + includeDepth(includeDepth), + importMachines(importMachines), + cur_token(0), + line(1), column(1), lastnl(0), + parser(0), ignoreSection(false), + parserExistsError(false), + whitespaceOn(true), + lastToken(0) + {} + + bool duplicateInclude( char *inclFileName, char *inclSectionName ); + + /* Make a list of places to look for an included file. */ + char **makeIncludePathChecks( const char *curFileName, const char *fileName, int len ); + std::ifstream *tryOpenInclude( char **pathChecks, long &found ); + + void handleMachine(); + void handleInclude(); + void handleImport(); + + void init(); + void token( int type, char *start, char *end ); + void token( int type, char c ); + void token( int type ); + void processToken( int type, char *tokdata, int toklen ); + void directToParser( Parser *toParser, const char *tokFileName, int tokLine, + int tokColumn, int type, char *tokdata, int toklen ); + void flushImport( ); + void importToken( int type, char *start, char *end ); + void pass( int token, char *start, char *end ); + void pass(); + void updateCol(); + void startSection(); + void endSection(); + void do_scan(); + bool active(); + ostream &scan_error(); + + InputData &id; + const char *fileName; + istream &input; + Parser *inclToParser; + char *inclSectionTarg; + int includeDepth; + bool importMachines; + + /* For import parsing. */ + int tok_cs, tok_act; + int *tok_ts, *tok_te; + int cur_token; + static const int max_tokens = 32; + int token_data[max_tokens]; + char *token_strings[max_tokens]; + int token_lens[max_tokens]; + + /* For section processing. */ + int cs; + char *word, *lit; + int word_len, lit_len; + + /* For character scanning. */ + int line; + InputLoc sectionLoc; + char *ts, *te; + int column; + char *lastnl; + + /* Set by machine statements, these persist from section to section + * allowing for unnamed sections. */ + Parser *parser; + bool ignoreSection; + + /* This is set if ragel has already emitted an error stating that + * no section name has been seen and thus no parser exists. */ + bool parserExistsError; + + /* This is for inline code. By default it is on. It goes off for + * statements and values in inline blocks which are parsed. */ + bool whitespaceOn; + + /* Keeps a record of the previous token sent to the section parser. */ + int lastToken; +}; + +#endif diff --git a/contrib/tools/ragel6/rlscan.rl b/contrib/tools/ragel6/rlscan.rl new file mode 100644 index 0000000000..c3217f1542 --- /dev/null +++ b/contrib/tools/ragel6/rlscan.rl @@ -0,0 +1,1272 @@ +/* + * Copyright 2006-2007 Adrian Thurston <thurston@complang.org> + * Copyright 2011 Josef Goettgens + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iostream> +#include <fstream> +#include <string.h> + +#include "ragel.h" +#include "rlscan.h" +#include "inputdata.h" + +//#define LOG_TOKENS + +using std::ifstream; +using std::istream; +using std::ostream; +using std::cout; +using std::cerr; +using std::endl; + +enum InlineBlockType +{ + CurlyDelimited, + SemiTerminated +}; + + +/* + * The Scanner for Importing + */ + +%%{ + machine inline_token_scan; + alphtype int; + access tok_; + + # Import scanner tokens. + import "rlparse.h"; + + main := |* + # Define of number. + IMP_Define IMP_Word IMP_UInt => { + int base = tok_ts - token_data; + int nameOff = 1; + int numOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_UInt, + token_strings[base+numOff], token_lens[base+numOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }; + + # Assignment of number. + IMP_Word '=' IMP_UInt => { + int base = tok_ts - token_data; + int nameOff = 0; + int numOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_UInt, + token_strings[base+numOff], token_lens[base+numOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }; + + # Define of literal. + IMP_Define IMP_Word IMP_Literal => { + int base = tok_ts - token_data; + int nameOff = 1; + int litOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_Literal, + token_strings[base+litOff], token_lens[base+litOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }; + + # Assignment of literal. + IMP_Word '=' IMP_Literal => { + int base = tok_ts - token_data; + int nameOff = 0; + int litOff = 2; + + directToParser( inclToParser, fileName, line, column, TK_Word, + token_strings[base+nameOff], token_lens[base+nameOff] ); + directToParser( inclToParser, fileName, line, column, '=', 0, 0 ); + directToParser( inclToParser, fileName, line, column, TK_Literal, + token_strings[base+litOff], token_lens[base+litOff] ); + directToParser( inclToParser, fileName, line, column, ';', 0, 0 ); + }; + + # Catch everything else. + any; + *|; +}%% + +%% write data; + +void Scanner::flushImport() +{ + int *p = token_data; + int *pe = token_data + cur_token; + int *eof = 0; + + %%{ + machine inline_token_scan; + write init; + write exec; + }%% + + if ( tok_ts == 0 ) + cur_token = 0; + else { + cur_token = pe - tok_ts; + int ts_offset = tok_ts - token_data; + memmove( token_data, token_data+ts_offset, cur_token*sizeof(token_data[0]) ); + memmove( token_strings, token_strings+ts_offset, cur_token*sizeof(token_strings[0]) ); + memmove( token_lens, token_lens+ts_offset, cur_token*sizeof(token_lens[0]) ); + } +} + +void Scanner::directToParser( Parser *toParser, const char *tokFileName, int tokLine, + int tokColumn, int type, char *tokdata, int toklen ) +{ + InputLoc loc; + + #ifdef LOG_TOKENS + cerr << "scanner:" << tokLine << ":" << tokColumn << + ": sending token to the parser " << Parser_lelNames[type]; + cerr << " " << toklen; + if ( tokdata != 0 ) + cerr << " " << tokdata; + cerr << endl; + #endif + + loc.fileName = tokFileName; + loc.line = tokLine; + loc.col = tokColumn; + + toParser->token( loc, type, tokdata, toklen ); +} + +void Scanner::importToken( int token, char *start, char *end ) +{ + if ( cur_token == max_tokens ) + flushImport(); + + token_data[cur_token] = token; + if ( start == 0 ) { + token_strings[cur_token] = 0; + token_lens[cur_token] = 0; + } + else { + int toklen = end-start; + token_lens[cur_token] = toklen; + token_strings[cur_token] = new char[toklen+1]; + memcpy( token_strings[cur_token], start, toklen ); + token_strings[cur_token][toklen] = 0; + } + cur_token++; +} + +void Scanner::pass( int token, char *start, char *end ) +{ + if ( importMachines ) + importToken( token, start, end ); + pass(); +} + +void Scanner::pass() +{ + updateCol(); + + /* If no errors and we are at the bottom of the include stack (the + * source file listed on the command line) then write out the data. */ + if ( includeDepth == 0 && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->data.write( ts, te-ts ); +} + +/* + * The scanner for processing sections, includes, imports, etc. + */ + +%%{ + machine section_parse; + alphtype int; + write data; +}%% + + +void Scanner::init( ) +{ + %% write init; +} + +bool Scanner::active() +{ + if ( ignoreSection ) + return false; + + if ( parser == 0 && ! parserExistsError ) { + scan_error() << "this specification has no name, nor does any previous" + " specification" << endl; + parserExistsError = true; + } + + if ( parser == 0 ) + return false; + + return true; +} + +ostream &Scanner::scan_error() +{ + /* Maintain the error count. */ + gblErrorCount += 1; + cerr << makeInputLoc( fileName, line, column ) << ": "; + return cerr; +} + +/* An approximate check for duplicate includes. Due to aliasing of files it's + * possible for duplicates to creep in. */ +bool Scanner::duplicateInclude( char *inclFileName, char *inclSectionName ) +{ + for ( IncludeHistory::Iter hi = parser->includeHistory; hi.lte(); hi++ ) { + if ( strcmp( hi->fileName, inclFileName ) == 0 && + strcmp( hi->sectionName, inclSectionName ) == 0 ) + { + return true; + } + } + return false; +} + +void Scanner::updateCol() +{ + char *from = lastnl; + if ( from == 0 ) + from = ts; + //cerr << "adding " << te - from << " to column" << endl; + column += te - from; + lastnl = 0; +} + +void Scanner::handleMachine() +{ + /* Assign a name to the machine. */ + char *machine = word; + + if ( !importMachines && inclSectionTarg == 0 ) { + ignoreSection = false; + + ParserDictEl *pdEl = id.parserDict.find( machine ); + if ( pdEl == 0 ) { + pdEl = new ParserDictEl( machine ); + pdEl->value = new Parser( fileName, machine, sectionLoc ); + pdEl->value->init(); + id.parserDict.insert( pdEl ); + id.parserList.append( pdEl->value ); + } + + parser = pdEl->value; + } + else if ( !importMachines && strcmp( inclSectionTarg, machine ) == 0 ) { + /* found include target */ + ignoreSection = false; + parser = inclToParser; + } + else { + /* ignoring section */ + ignoreSection = true; + parser = 0; + } +} + +void Scanner::handleInclude() +{ + if ( active() ) { + char *inclSectionName = word; + char **includeChecks = 0; + + /* Implement defaults for the input file and section name. */ + if ( inclSectionName == 0 ) + inclSectionName = parser->sectionName; + + if ( lit != 0 ) + includeChecks = makeIncludePathChecks( fileName, lit, lit_len ); + else { + char *test = new char[strlen(fileName)+1]; + strcpy( test, fileName ); + + includeChecks = new char*[2]; + + includeChecks[0] = test; + includeChecks[1] = 0; + } + + long found = 0; + ifstream *inFile = tryOpenInclude( includeChecks, found ); + if ( inFile == 0 ) { + scan_error() << "include: failed to locate file" << endl; + char **tried = includeChecks; + while ( *tried != 0 ) + scan_error() << "include: attempted: \"" << *tried++ << '\"' << endl; + } + else { + /* Don't include anything that's already been included. */ + if ( !duplicateInclude( includeChecks[found], inclSectionName ) ) { + parser->includeHistory.append( IncludeHistoryItem( + includeChecks[found], inclSectionName ) ); + + Scanner scanner( id, includeChecks[found], *inFile, parser, + inclSectionName, includeDepth+1, false ); + scanner.do_scan( ); + delete inFile; + } + } + } +} + +void Scanner::handleImport() +{ + if ( active() ) { + char **importChecks = makeIncludePathChecks( fileName, lit, lit_len ); + + /* Open the input file for reading. */ + long found = 0; + ifstream *inFile = tryOpenInclude( importChecks, found ); + if ( inFile == 0 ) { + scan_error() << "import: could not open import file " << + "for reading" << endl; + char **tried = importChecks; + while ( *tried != 0 ) + scan_error() << "import: attempted: \"" << *tried++ << '\"' << endl; + } + + Scanner scanner( id, importChecks[found], *inFile, parser, + 0, includeDepth+1, true ); + scanner.do_scan( ); + scanner.importToken( 0, 0, 0 ); + scanner.flushImport(); + delete inFile; + } +} + +%%{ + machine section_parse; + + # Need the defines representing tokens. + import "rlparse.h"; + + action clear_words { word = lit = 0; word_len = lit_len = 0; } + action store_word { word = tokdata; word_len = toklen; } + action store_lit { lit = tokdata; lit_len = toklen; } + + action mach_err { scan_error() << "bad machine statement" << endl; } + action incl_err { scan_error() << "bad include statement" << endl; } + action import_err { scan_error() << "bad import statement" << endl; } + action write_err { scan_error() << "bad write statement" << endl; } + + action handle_machine { handleMachine(); } + action handle_include { handleInclude(); } + action handle_import { handleImport(); } + + machine_stmt = + ( KW_Machine TK_Word @store_word ';' ) @handle_machine + <>err mach_err <>eof mach_err; + + include_names = ( + TK_Word @store_word ( TK_Literal @store_lit )? | + TK_Literal @store_lit + ) >clear_words; + + include_stmt = + ( KW_Include include_names ';' ) @handle_include + <>err incl_err <>eof incl_err; + + import_stmt = + ( KW_Import TK_Literal @store_lit ';' ) @handle_import + <>err import_err <>eof import_err; + + action write_command + { + if ( active() && machineSpec == 0 && machineName == 0 ) { + InputItem *inputItem = new InputItem; + inputItem->type = InputItem::Write; + inputItem->loc.fileName = fileName; + inputItem->loc.line = line; + inputItem->loc.col = column; + inputItem->name = parser->sectionName; + inputItem->pd = parser->pd; + id.inputItems.append( inputItem ); + } + } + + action write_arg + { + if ( active() && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->writeArgs.append( strdup(tokdata) ); + } + + action write_close + { + if ( active() && machineSpec == 0 && machineName == 0 ) + id.inputItems.tail->writeArgs.append( 0 ); + } + + write_stmt = + ( KW_Write @write_command + ( TK_Word @write_arg )+ ';' @write_close ) + <>err write_err <>eof write_err; + + action handle_token + { + /* Send the token off to the parser. */ + if ( active() ) + directToParser( parser, fileName, line, column, type, tokdata, toklen ); + } + + # Catch everything else. + everything_else = + ^( KW_Machine | KW_Include | KW_Import | KW_Write ) @handle_token; + + main := ( + machine_stmt | + include_stmt | + import_stmt | + write_stmt | + everything_else + )*; +}%% + +void Scanner::token( int type, char c ) +{ + token( type, &c, &c + 1 ); +} + +void Scanner::token( int type ) +{ + token( type, 0, 0 ); +} + +void Scanner::token( int type, char *start, char *end ) +{ + char *tokdata = 0; + int toklen = 0; + if ( start != 0 ) { + toklen = end-start; + tokdata = new char[toklen+1]; + memcpy( tokdata, start, toklen ); + tokdata[toklen] = 0; + } + + processToken( type, tokdata, toklen ); +} + +void Scanner::processToken( int type, char *tokdata, int toklen ) +{ + int *p, *pe, *eof; + + if ( type < 0 ) + p = pe = eof = 0; + else { + p = &type; + pe = &type + 1; + eof = 0; + } + + %%{ + machine section_parse; + write exec; + }%% + + updateCol(); + + /* Record the last token for use in controlling the scan of subsequent + * tokens. */ + lastToken = type; +} + +void Scanner::startSection( ) +{ + parserExistsError = false; + + sectionLoc.fileName = fileName; + sectionLoc.line = line; + sectionLoc.col = column; +} + +void Scanner::endSection( ) +{ + /* Execute the eof actions for the section parser. */ + processToken( -1, 0, 0 ); + + /* Close off the section with the parser. */ + if ( active() ) { + InputLoc loc; + loc.fileName = fileName; + loc.line = line; + loc.col = column; + + parser->token( loc, TK_EndSection, 0, 0 ); + } + + if ( includeDepth == 0 ) { + if ( machineSpec == 0 && machineName == 0 ) { + /* The end section may include a newline on the end, so + * we use the last line, which will count the newline. */ + InputItem *inputItem = new InputItem; + inputItem->type = InputItem::HostData; + inputItem->loc.line = line; + inputItem->loc.col = column; + id.inputItems.append( inputItem ); + } + } +} + +bool isAbsolutePath( const char *path ) +{ +#ifdef _WIN32 + return isalpha( path[0] ) && path[1] == ':' && (path[2] == '\\' || path[2] == '/'); +#else + return path[0] == '/'; +#endif +} + +inline char* resolvePath(const char* rel, const char* abs) { + const size_t l1 = strlen(rel); + const size_t l2 = strlen(abs); + char* ret = new char[l1 + l2 + 1]; + + const char* p = strrchr(abs, '/') + 1; + const size_t l3 = p - abs; + + memcpy(ret, abs, l3); + strcpy(ret + l3, rel); + + return ret; +} + +char **Scanner::makeIncludePathChecks( const char *thisFileName, + const char *fileName, int fnlen ) +{ + char **checks = 0; + long nextCheck = 0; + long length = 0; + bool caseInsensitive = false; + char *data = prepareLitString( InputLoc(), fileName, fnlen, + length, caseInsensitive ); + + /* Absolute path? */ + if ( isAbsolutePath( data ) ) { + checks = new char*[2]; + checks[nextCheck++] = data; + } + else { + checks = new char*[2 + id.includePaths.length()]; + + /* Search from the the location of the current file. */ + const char *lastSlash = strrchr( thisFileName, '/' ); + if ( lastSlash == 0 ) + checks[nextCheck++] = data; + else { + checks[nextCheck++] = resolvePath(data, thisFileName); + } + + /* Search from the include paths given on the command line. */ + for ( ArgsVector::Iter incp = id.includePaths; incp.lte(); incp++ ) { + long pathLen = strlen( *incp ); + long checkLen = pathLen + 1 + length; + char *check = new char[checkLen+1]; + memcpy( check, *incp, pathLen ); + check[pathLen] = '/'; + memcpy( check+pathLen+1, data, length ); + check[checkLen] = 0; + checks[nextCheck++] = check; + } + } + + checks[nextCheck] = 0; + return checks; +} + +ifstream *Scanner::tryOpenInclude( char **pathChecks, long &found ) +{ + char **check = pathChecks; + ifstream *inFile = new ifstream; + + while ( *check != 0 ) { + inFile->open( *check ); + if ( inFile->is_open() ) { + found = check - pathChecks; + return inFile; + } + + /* + * 03/26/2011 jg: + * Don't rely on sloppy runtime behaviour: reset the state of the stream explicitly. + * If inFile->open() fails, which happens when include dirs are tested, the fail bit + * is set by the runtime library. Currently the VS runtime library opens new files, + * but when it comes to reading it refuses to work. + */ + inFile->clear(); + + check += 1; + } + + found = -1; + delete inFile; + return 0; +} + +%%{ + machine rlscan; + + # This is sent by the driver code. + EOF = 0; + + action inc_nl { + lastnl = p; + column = 0; + line++; + } + NL = '\n' @inc_nl; + + # Identifiers, numbers, commetns, and other common things. + ident = ( alpha | '_' ) ( alpha |digit |'_' )*; + number = digit+; + hex_number = '0x' [0-9a-fA-F]+; + + c_comment = + '/*' ( any | NL )* :>> '*/'; + + cpp_comment = + '//' [^\n]* NL; + + c_cpp_comment = c_comment | cpp_comment; + + ruby_comment = '#' [^\n]* NL; + + # These literal forms are common to host code and ragel. + s_literal = "'" ([^'\\] | NL | '\\' (any | NL))* "'"; + d_literal = '"' ([^"\\] | NL | '\\' (any | NL))* '"'; + host_re_literal = '/' ([^/\\] | NL | '\\' (any | NL))* '/'; + + whitespace = [ \t] | NL; + pound_comment = '#' [^\n]* NL; + + # An inline block of code for Ruby. + inline_code_ruby := |* + # Inline expression keywords. + "fpc" => { token( KW_PChar ); }; + "fc" => { token( KW_Char ); }; + "fcurs" => { token( KW_CurState ); }; + "ftargs" => { token( KW_TargState ); }; + "fentry" => { + whitespaceOn = false; + token( KW_Entry ); + }; + + # Inline statement keywords. + "fhold" => { + whitespaceOn = false; + token( KW_Hold ); + }; + "fexec" => { token( KW_Exec, 0, 0 ); }; + "fgoto" => { + whitespaceOn = false; + token( KW_Goto ); + }; + "fnext" => { + whitespaceOn = false; + token( KW_Next ); + }; + "fcall" => { + whitespaceOn = false; + token( KW_Call ); + }; + "fret" => { + whitespaceOn = false; + token( KW_Ret ); + }; + "fbreak" => { + whitespaceOn = false; + token( KW_Break ); + }; + + ident => { token( TK_Word, ts, te ); }; + + number => { token( TK_UInt, ts, te ); }; + hex_number => { token( TK_Hex, ts, te ); }; + + ( s_literal | d_literal | host_re_literal ) + => { token( IL_Literal, ts, te ); }; + + whitespace+ => { + if ( whitespaceOn ) + token( IL_WhiteSpace, ts, te ); + }; + + ruby_comment => { token( IL_Comment, ts, te ); }; + + "::" => { token( TK_NameSep, ts, te ); }; + + # Some symbols need to go to the parser as with their cardinal value as + # the token type (as opposed to being sent as anonymous symbols) + # because they are part of the sequences which we interpret. The * ) ; + # symbols cause whitespace parsing to come back on. This gets turned + # off by some keywords. + + ";" => { + whitespaceOn = true; + token( *ts, ts, te ); + if ( inlineBlockType == SemiTerminated ) + fret; + }; + + [*)] => { + whitespaceOn = true; + token( *ts, ts, te ); + }; + + [,(] => { token( *ts, ts, te ); }; + + '{' => { + token( IL_Symbol, ts, te ); + curly_count += 1; + }; + + '}' => { + if ( --curly_count == 0 && inlineBlockType == CurlyDelimited ) { + /* Inline code block ends. */ + token( '}' ); + fret; + } + else { + /* Either a semi terminated inline block or only the closing + * brace of some inner scope, not the block's closing brace. */ + token( IL_Symbol, ts, te ); + } + }; + + EOF => { + scan_error() << "unterminated code block" << endl; + }; + + # Send every other character as a symbol. + any => { token( IL_Symbol, ts, te ); }; + *|; + + + # An inline block of code for languages other than Ruby. + inline_code := |* + # Inline expression keywords. + "fpc" => { token( KW_PChar ); }; + "fc" => { token( KW_Char ); }; + "fcurs" => { token( KW_CurState ); }; + "ftargs" => { token( KW_TargState ); }; + "fentry" => { + whitespaceOn = false; + token( KW_Entry ); + }; + + # Inline statement keywords. + "fhold" => { + whitespaceOn = false; + token( KW_Hold ); + }; + "fexec" => { token( KW_Exec, 0, 0 ); }; + "fgoto" => { + whitespaceOn = false; + token( KW_Goto ); + }; + "fnext" => { + whitespaceOn = false; + token( KW_Next ); + }; + "fcall" => { + whitespaceOn = false; + token( KW_Call ); + }; + "fret" => { + whitespaceOn = false; + token( KW_Ret ); + }; + "fbreak" => { + whitespaceOn = false; + token( KW_Break ); + }; + + ident => { token( TK_Word, ts, te ); }; + + number => { token( TK_UInt, ts, te ); }; + hex_number => { token( TK_Hex, ts, te ); }; + + ( s_literal | d_literal ) + => { token( IL_Literal, ts, te ); }; + + whitespace+ => { + if ( whitespaceOn ) + token( IL_WhiteSpace, ts, te ); + }; + + c_cpp_comment => { token( IL_Comment, ts, te ); }; + + "::" => { token( TK_NameSep, ts, te ); }; + + # Some symbols need to go to the parser as with their cardinal value as + # the token type (as opposed to being sent as anonymous symbols) + # because they are part of the sequences which we interpret. The * ) ; + # symbols cause whitespace parsing to come back on. This gets turned + # off by some keywords. + + ";" => { + whitespaceOn = true; + token( *ts, ts, te ); + if ( inlineBlockType == SemiTerminated ) + fret; + }; + + [*)] => { + whitespaceOn = true; + token( *ts, ts, te ); + }; + + [,(] => { token( *ts, ts, te ); }; + + '{' => { + token( IL_Symbol, ts, te ); + curly_count += 1; + }; + + '}' => { + if ( --curly_count == 0 && inlineBlockType == CurlyDelimited ) { + /* Inline code block ends. */ + token( '}' ); + fret; + } + else { + /* Either a semi terminated inline block or only the closing + * brace of some inner scope, not the block's closing brace. */ + token( IL_Symbol, ts, te ); + } + }; + + EOF => { + scan_error() << "unterminated code block" << endl; + }; + + # Send every other character as a symbol. + any => { token( IL_Symbol, ts, te ); }; + *|; + + or_literal := |* + # Escape sequences in OR expressions. + '\\0' => { token( RE_Char, '\0' ); }; + '\\a' => { token( RE_Char, '\a' ); }; + '\\b' => { token( RE_Char, '\b' ); }; + '\\t' => { token( RE_Char, '\t' ); }; + '\\n' => { token( RE_Char, '\n' ); }; + '\\v' => { token( RE_Char, '\v' ); }; + '\\f' => { token( RE_Char, '\f' ); }; + '\\r' => { token( RE_Char, '\r' ); }; + '\\\n' => { updateCol(); }; + '\\' any => { token( RE_Char, ts+1, te ); }; + + # Range dash in an OR expression. + '-' => { token( RE_Dash, 0, 0 ); }; + + # Terminate an OR expression. + ']' => { token( RE_SqClose ); fret; }; + + EOF => { + scan_error() << "unterminated OR literal" << endl; + }; + + # Characters in an OR expression. + [^\]] => { token( RE_Char, ts, te ); }; + + *|; + + ragel_re_literal := |* + # Escape sequences in regular expressions. + '\\0' => { token( RE_Char, '\0' ); }; + '\\a' => { token( RE_Char, '\a' ); }; + '\\b' => { token( RE_Char, '\b' ); }; + '\\t' => { token( RE_Char, '\t' ); }; + '\\n' => { token( RE_Char, '\n' ); }; + '\\v' => { token( RE_Char, '\v' ); }; + '\\f' => { token( RE_Char, '\f' ); }; + '\\r' => { token( RE_Char, '\r' ); }; + '\\\n' => { updateCol(); }; + '\\' any => { token( RE_Char, ts+1, te ); }; + + # Terminate an OR expression. + '/' [i]? => { + token( RE_Slash, ts, te ); + fgoto parser_def; + }; + + # Special characters. + '.' => { token( RE_Dot ); }; + '*' => { token( RE_Star ); }; + + '[' => { token( RE_SqOpen ); fcall or_literal; }; + '[^' => { token( RE_SqOpenNeg ); fcall or_literal; }; + + EOF => { + scan_error() << "unterminated regular expression" << endl; + }; + + # Characters in an OR expression. + [^\/] => { token( RE_Char, ts, te ); }; + *|; + + # We need a separate token space here to avoid the ragel keywords. + write_statement := |* + ident => { token( TK_Word, ts, te ); } ; + [ \t\n]+ => { updateCol(); }; + ';' => { token( ';' ); fgoto parser_def; }; + + EOF => { + scan_error() << "unterminated write statement" << endl; + }; + *|; + + # Parser definitions. + parser_def := |* + #'length_cond' => { token( KW_Length ); }; + 'machine' => { token( KW_Machine ); }; + 'include' => { token( KW_Include ); }; + 'import' => { token( KW_Import ); }; + 'write' => { + token( KW_Write ); + fgoto write_statement; + }; + 'action' => { token( KW_Action ); }; + 'alphtype' => { token( KW_AlphType ); }; + 'prepush' => { token( KW_PrePush ); }; + 'postpop' => { token( KW_PostPop ); }; + + # FIXME: Enable this post 5.17. + # 'range' => { token( KW_Range ); }; + + 'getkey' => { + token( KW_GetKey ); + inlineBlockType = SemiTerminated; + if ( hostLang->lang == HostLang::Ruby ) + fcall inline_code_ruby; + else + fcall inline_code; + }; + 'access' => { + token( KW_Access ); + inlineBlockType = SemiTerminated; + if ( hostLang->lang == HostLang::Ruby ) + fcall inline_code_ruby; + else + fcall inline_code; + }; + 'variable' => { + token( KW_Variable ); + inlineBlockType = SemiTerminated; + if ( hostLang->lang == HostLang::Ruby ) + fcall inline_code_ruby; + else + fcall inline_code; + }; + 'when' => { token( KW_When ); }; + 'inwhen' => { token( KW_InWhen ); }; + 'outwhen' => { token( KW_OutWhen ); }; + 'eof' => { token( KW_Eof ); }; + 'err' => { token( KW_Err ); }; + 'lerr' => { token( KW_Lerr ); }; + 'to' => { token( KW_To ); }; + 'from' => { token( KW_From ); }; + 'export' => { token( KW_Export ); }; + + # Identifiers. + ident => { token( TK_Word, ts, te ); } ; + + # Numbers + number => { token( TK_UInt, ts, te ); }; + hex_number => { token( TK_Hex, ts, te ); }; + + # Literals, with optionals. + ( s_literal | d_literal ) [i]? + => { token( TK_Literal, ts, te ); }; + + '[' => { token( RE_SqOpen ); fcall or_literal; }; + '[^' => { token( RE_SqOpenNeg ); fcall or_literal; }; + + '/' => { token( RE_Slash ); fgoto ragel_re_literal; }; + + # Ignore. + pound_comment => { updateCol(); }; + + ':=' => { token( TK_ColonEquals ); }; + + # To State Actions. + ">~" => { token( TK_StartToState ); }; + "$~" => { token( TK_AllToState ); }; + "%~" => { token( TK_FinalToState ); }; + "<~" => { token( TK_NotStartToState ); }; + "@~" => { token( TK_NotFinalToState ); }; + "<>~" => { token( TK_MiddleToState ); }; + + # From State actions + ">*" => { token( TK_StartFromState ); }; + "$*" => { token( TK_AllFromState ); }; + "%*" => { token( TK_FinalFromState ); }; + "<*" => { token( TK_NotStartFromState ); }; + "@*" => { token( TK_NotFinalFromState ); }; + "<>*" => { token( TK_MiddleFromState ); }; + + # EOF Actions. + ">/" => { token( TK_StartEOF ); }; + "$/" => { token( TK_AllEOF ); }; + "%/" => { token( TK_FinalEOF ); }; + "</" => { token( TK_NotStartEOF ); }; + "@/" => { token( TK_NotFinalEOF ); }; + "<>/" => { token( TK_MiddleEOF ); }; + + # Global Error actions. + ">!" => { token( TK_StartGblError ); }; + "$!" => { token( TK_AllGblError ); }; + "%!" => { token( TK_FinalGblError ); }; + "<!" => { token( TK_NotStartGblError ); }; + "@!" => { token( TK_NotFinalGblError ); }; + "<>!" => { token( TK_MiddleGblError ); }; + + # Local error actions. + ">^" => { token( TK_StartLocalError ); }; + "$^" => { token( TK_AllLocalError ); }; + "%^" => { token( TK_FinalLocalError ); }; + "<^" => { token( TK_NotStartLocalError ); }; + "@^" => { token( TK_NotFinalLocalError ); }; + "<>^" => { token( TK_MiddleLocalError ); }; + + # Middle. + "<>" => { token( TK_Middle ); }; + + # Conditions. + '>?' => { token( TK_StartCond ); }; + '$?' => { token( TK_AllCond ); }; + '%?' => { token( TK_LeavingCond ); }; + + '..' => { token( TK_DotDot ); }; + '**' => { token( TK_StarStar ); }; + '--' => { token( TK_DashDash ); }; + '->' => { token( TK_Arrow ); }; + '=>' => { token( TK_DoubleArrow ); }; + + ":>" => { token( TK_ColonGt ); }; + ":>>" => { token( TK_ColonGtGt ); }; + "<:" => { token( TK_LtColon ); }; + + # Opening of longest match. + "|*" => { token( TK_BarStar ); }; + + # Separater for name references. + "::" => { token( TK_NameSep, ts, te ); }; + + '}%%' => { + updateCol(); + endSection(); + fret; + }; + + [ \t\r]+ => { updateCol(); }; + + # If we are in a single line machine then newline may end the spec. + NL => { + updateCol(); + if ( singleLineSpec ) { + endSection(); + fret; + } + }; + + '{' => { + if ( lastToken == KW_Export || lastToken == KW_Entry ) + token( '{' ); + else { + token( '{' ); + curly_count = 1; + inlineBlockType = CurlyDelimited; + if ( hostLang->lang == HostLang::Ruby ) + fcall inline_code_ruby; + else + fcall inline_code; + } + }; + + EOF => { + scan_error() << "unterminated ragel section" << endl; + }; + + any => { token( *ts ); } ; + *|; + + # Outside code scanner. These tokens get passed through. + main_ruby := |* + ident => { pass( IMP_Word, ts, te ); }; + number => { pass( IMP_UInt, ts, te ); }; + ruby_comment => { pass(); }; + ( s_literal | d_literal | host_re_literal ) + => { pass( IMP_Literal, ts, te ); }; + + '%%{' => { + updateCol(); + singleLineSpec = false; + startSection(); + fcall parser_def; + }; + '%%' => { + updateCol(); + singleLineSpec = true; + startSection(); + fcall parser_def; + }; + whitespace+ => { pass(); }; + EOF; + any => { pass( *ts, 0, 0 ); }; + *|; + + # Outside code scanner. These tokens get passed through. + main := |* + 'define' => { pass( IMP_Define, 0, 0 ); }; + ident => { pass( IMP_Word, ts, te ); }; + number => { pass( IMP_UInt, ts, te ); }; + c_cpp_comment => { pass(); }; + ( s_literal | d_literal ) => { pass( IMP_Literal, ts, te ); }; + + '%%{' => { + updateCol(); + singleLineSpec = false; + startSection(); + fcall parser_def; + }; + '%%' => { + updateCol(); + singleLineSpec = true; + startSection(); + fcall parser_def; + }; + whitespace+ => { pass(); }; + EOF; + any => { pass( *ts, 0, 0 ); }; + *|; +}%% + +%% write data; + +void Scanner::do_scan() +{ + int bufsize = 8; + char *buf = new char[bufsize]; + int cs, act, have = 0; + int top; + + /* The stack is two deep, one level for going into ragel defs from the main + * machines which process outside code, and another for going into or literals + * from either a ragel spec, or a regular expression. */ + int stack[2]; + int curly_count = 0; + bool execute = true; + bool singleLineSpec = false; + InlineBlockType inlineBlockType = CurlyDelimited; + + /* Init the section parser and the character scanner. */ + init(); + %% write init; + + /* Set up the start state. FIXME: After 5.20 is released the nocs write + * init option should be used, the main machine eliminated and this statement moved + * above the write init. */ + if ( hostLang->lang == HostLang::Ruby ) + cs = rlscan_en_main_ruby; + else + cs = rlscan_en_main; + + while ( execute ) { + char *p = buf + have; + int space = bufsize - have; + + if ( space == 0 ) { + /* We filled up the buffer trying to scan a token. Grow it. */ + bufsize = bufsize * 2; + char *newbuf = new char[bufsize]; + + /* Recompute p and space. */ + p = newbuf + have; + space = bufsize - have; + + /* Patch up pointers possibly in use. */ + if ( ts != 0 ) + ts = newbuf + ( ts - buf ); + te = newbuf + ( te - buf ); + + /* Copy the new buffer in. */ + memcpy( newbuf, buf, have ); + delete[] buf; + buf = newbuf; + } + + input.read( p, space ); + int len = input.gcount(); + char *pe = p + len; + + /* If we see eof then append the eof var. */ + char *eof = 0; + if ( len == 0 ) { + eof = pe; + execute = false; + } + + %% write exec; + + /* Check if we failed. */ + if ( cs == rlscan_error ) { + /* Machine failed before finding a token. I'm not yet sure if this + * is reachable. */ + scan_error() << "scanner error" << endl; + exit(1); + } + + /* Decide if we need to preserve anything. */ + char *preserve = ts; + + /* Now set up the prefix. */ + if ( preserve == 0 ) + have = 0; + else { + /* There is data that needs to be shifted over. */ + have = pe - preserve; + memmove( buf, preserve, have ); + unsigned int shiftback = preserve - buf; + if ( ts != 0 ) + ts -= shiftback; + te -= shiftback; + + preserve = buf; + } + } + + delete[] buf; +} diff --git a/contrib/tools/ragel6/rubycodegen.cpp b/contrib/tools/ragel6/rubycodegen.cpp new file mode 100644 index 0000000000..ee68238934 --- /dev/null +++ b/contrib/tools/ragel6/rubycodegen.cpp @@ -0,0 +1,831 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iomanip> +#include <sstream> +#include "redfsm.h" +#include "gendata.h" +#include "ragel.h" +#include "rubycodegen.h" +#include "pcheck.h" +#include "vector.h" +#include "version.h" +#include "common.h" + +#include "ragel.h" +#include "rubytable.h" +#include "rubyftable.h" +#include "rubyflat.h" +#include "rubyfflat.h" +#include "rbxgoto.h" + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; +using std::istream; +using std::ifstream; +using std::ostream; +using std::ios; +using std::cin; +using std::cout; +using std::cerr; +using std::endl; + +/* Target ruby impl */ + +/* Target language and output style. */ +extern CodeStyle codeStyle; + +extern int numSplitPartitions; +extern bool noLineDirectives; + +/* + * Callbacks invoked by the XML data parser. + */ + + +void rubyLineDirective( ostream &out, const char *fileName, int line ) +{ + if ( noLineDirectives ) + return; + + /* Write a comment containing line info. */ + out << "# line " << line << " \""; + for ( const char *pc = fileName; *pc != 0; pc++ ) { + if ( *pc == '\\' ) + out << "\\\\"; + else + out << *pc; + } + out << "\"\n"; +} + +void RubyCodeGen::genLineDirective( ostream &out ) +{ + std::streambuf *sbuf = out.rdbuf(); + output_filter *filter = static_cast<output_filter*>(sbuf); + rubyLineDirective( out, filter->fileName, filter->line + 1 ); +} + +string RubyCodeGen::DATA_PREFIX() +{ + if ( !noPrefix ) + return FSM_NAME() + "_"; + return ""; +} + +std::ostream &RubyCodeGen::STATIC_VAR( string type, string name ) +{ + out << + "class << self\n" + " attr_accessor :" << name << "\n" + "end\n" + "self." << name; + return out; +} + + +std::ostream &RubyCodeGen::OPEN_ARRAY( string type, string name ) +{ + out << + "class << self\n" + " attr_accessor :" << name << "\n" + " private :" << name << ", :" << name << "=\n" + "end\n" + "self." << name << " = [\n"; + return out; +} + +std::ostream &RubyCodeGen::CLOSE_ARRAY() +{ + out << "]\n"; + return out; +} + + +string RubyCodeGen::ARR_OFF( string ptr, string offset ) +{ + return ptr + "[" + offset + "]"; +} + +string RubyCodeGen::NULL_ITEM() +{ + return "nil"; +} + + +string RubyCodeGen::P() +{ + ostringstream ret; + if ( pExpr == 0 ) + ret << "p"; + else { + //ret << "("; + INLINE_LIST( ret, pExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::PE() +{ + ostringstream ret; + if ( peExpr == 0 ) + ret << "pe"; + else { + //ret << "("; + INLINE_LIST( ret, peExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::vEOF() +{ + ostringstream ret; + if ( eofExpr == 0 ) + ret << "eof"; + else { + //ret << "("; + INLINE_LIST( ret, eofExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::vCS() +{ + ostringstream ret; + if ( csExpr == 0 ) + ret << ACCESS() << "cs"; + else { + //ret << "("; + INLINE_LIST( ret, csExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::TOP() +{ + ostringstream ret; + if ( topExpr == 0 ) + ret << ACCESS() + "top"; + else { + //ret << "("; + INLINE_LIST( ret, topExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::STACK() +{ + ostringstream ret; + if ( stackExpr == 0 ) + ret << ACCESS() + "stack"; + else { + //ret << "("; + INLINE_LIST( ret, stackExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::ACT() +{ + ostringstream ret; + if ( actExpr == 0 ) + ret << ACCESS() + "act"; + else { + //ret << "("; + INLINE_LIST( ret, actExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::TOKSTART() +{ + ostringstream ret; + if ( tokstartExpr == 0 ) + ret << ACCESS() + "ts"; + else { + //ret << "("; + INLINE_LIST( ret, tokstartExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::TOKEND() +{ + ostringstream ret; + if ( tokendExpr == 0 ) + ret << ACCESS() + "te"; + else { + //ret << "("; + INLINE_LIST( ret, tokendExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +string RubyCodeGen::DATA() +{ + ostringstream ret; + if ( dataExpr == 0 ) + ret << ACCESS() + "data"; + else { + //ret << "("; + INLINE_LIST( ret, dataExpr, 0, false ); + //ret << ")"; + } + return ret.str(); +} + +/* Write out the fsm name. */ +string RubyCodeGen::FSM_NAME() +{ + return fsmName; +} + + +void RubyCodeGen::ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ) +{ + /* Write the preprocessor line info for going into the source file. */ + rubyLineDirective( ret, action->loc.fileName, action->loc.line ); + + /* Write the block and close it off. */ + ret << " begin\n"; + INLINE_LIST( ret, action->inlineList, targState, inFinish ); + ret << " end\n"; +} + + + +string RubyCodeGen::GET_WIDE_KEY() +{ + if ( redFsm->anyConditions() ) + return "_widec"; + else + return GET_KEY(); +} + +string RubyCodeGen::GET_WIDE_KEY( RedStateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) + return "_widec"; + else + return GET_KEY(); +} + +string RubyCodeGen::GET_KEY() +{ + ostringstream ret; + if ( getKeyExpr != 0 ) { + /* Emit the user supplied method of retrieving the key. */ + ret << "("; + INLINE_LIST( ret, getKeyExpr, 0, false ); + ret << ")"; + } + else { + /* Expression for retrieving the key, use dereference and read ordinal, + * for compatibility with Ruby 1.9. */ + ret << DATA() << "[" << P() << "].ord"; + } + return ret.str(); +} + +string RubyCodeGen::KEY( Key key ) +{ + ostringstream ret; + if ( keyOps->isSigned || !hostLang->explicitUnsigned ) + ret << key.getVal(); + else + ret << (unsigned long) key.getVal(); + return ret.str(); +} + + +/* Write out level number of tabs. Makes the nested binary search nice + * looking. */ +string RubyCodeGen::TABS( int level ) +{ + string result; + while ( level-- > 0 ) + result += "\t"; + return result; +} + +string RubyCodeGen::INT( int i ) +{ + ostringstream ret; + ret << i; + return ret.str(); +} + +void RubyCodeGen::CONDITION( ostream &ret, GenAction *condition ) +{ + ret << "\n"; + rubyLineDirective( ret, condition->loc.fileName, condition->loc.line ); + INLINE_LIST( ret, condition->inlineList, 0, false ); +} + +/* Emit the alphabet data type. */ +string RubyCodeGen::ALPH_TYPE() +{ + string ret = keyOps->alphType->data1; + if ( keyOps->alphType->data2 != 0 ) { + ret += " "; + ret += + keyOps->alphType->data2; + } + return ret; +} + +/* Emit the alphabet data type. */ +string RubyCodeGen::WIDE_ALPH_TYPE() +{ + string ret; + if ( redFsm->maxKey <= keyOps->maxKey ) + ret = ALPH_TYPE(); + else { + long long maxKeyVal = redFsm->maxKey.getLongLong(); + HostType *wideType = keyOps->typeSubsumes( keyOps->isSigned, maxKeyVal ); + assert( wideType != 0 ); + + ret = wideType->data1; + if ( wideType->data2 != 0 ) { + ret += " "; + ret += wideType->data2; + } + } + return ret; +} + + +string RubyCodeGen::ARRAY_TYPE( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + + string ret = arrayType->data1; + if ( arrayType->data2 != 0 ) { + ret += " "; + ret += arrayType->data2; + } + return ret; +} + +/* Write out the array of actions. */ +std::ostream &RubyCodeGen::ACTIONS_ARRAY() +{ + START_ARRAY_LINE(); + int totalActions = 0; + ARRAY_ITEM( INT(0), ++totalActions, false ); + for ( GenActionTableMap::Iter act = redFsm->actionMap; act.lte(); act++ ) { + /* Write out the length, which will never be the last character. */ + ARRAY_ITEM( INT(act->key.length()), ++totalActions, false ); + + for ( GenActionTable::Iter item = act->key; item.lte(); item++ ) { + ARRAY_ITEM( INT(item->value->actionId), ++totalActions, (act.last() && item.last()) ); + } + } + END_ARRAY_LINE(); + return out; +} + +void RubyCodeGen::STATE_IDS() +{ + if ( redFsm->startState != 0 ) + STATIC_VAR( "int", START() ) << " = " << START_STATE_ID() << ";\n"; + + if ( !noFinal ) + STATIC_VAR( "int" , FIRST_FINAL() ) << " = " << FIRST_FINAL_STATE() << ";\n"; + + if ( !noError ) + STATIC_VAR( "int", ERROR() ) << " = " << ERROR_STATE() << ";\n"; + + out << "\n"; + + if ( !noEntry && entryPointNames.length() > 0 ) { + for ( EntryNameVect::Iter en = entryPointNames; en.lte(); en++ ) { + STATIC_VAR( "int", DATA_PREFIX() + "en_" + *en ) << + " = " << entryPointIds[en.pos()] << ";\n"; + } + out << "\n"; + } +} + +std::ostream &RubyCodeGen::START_ARRAY_LINE() +{ + out << "\t"; + return out; +} + +std::ostream &RubyCodeGen::ARRAY_ITEM( string item, int count, bool last ) +{ + out << item; + if ( !last ) + { + out << ", "; + if ( count % IALL == 0 ) + { + END_ARRAY_LINE(); + START_ARRAY_LINE(); + } + } + return out; +} + +std::ostream &RubyCodeGen::END_ARRAY_LINE() +{ + out << "\n"; + return out; +} + +/* Emit the offset of the start state as a decimal integer. */ +string RubyCodeGen::START_STATE_ID() +{ + ostringstream ret; + ret << redFsm->startState->id; + return ret.str(); +}; + +string RubyCodeGen::ERROR_STATE() +{ + ostringstream ret; + if ( redFsm->errState != 0 ) + ret << redFsm->errState->id; + else + ret << "-1"; + return ret.str(); +} + +string RubyCodeGen::FIRST_FINAL_STATE() +{ + ostringstream ret; + if ( redFsm->firstFinState != 0 ) + ret << redFsm->firstFinState->id; + else + ret << redFsm->nextStateId; + return ret.str(); +} + +string RubyCodeGen::ACCESS() +{ + ostringstream ret; + if ( accessExpr != 0 ) + INLINE_LIST( ret, accessExpr, 0, false ); + return ret.str(); +} + +/* Write out an inline tree structure. Walks the list and possibly calls out + * to virtual functions than handle language specific items in the tree. */ +void RubyCodeGen::INLINE_LIST( ostream &ret, GenInlineList *inlineList, + int targState, bool inFinish ) +{ + for ( GenInlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case GenInlineItem::Text: + ret << item->data; + break; + case GenInlineItem::Goto: + GOTO( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Call: + CALL( ret, item->targState->id, targState, inFinish ); + break; + case GenInlineItem::Next: + NEXT( ret, item->targState->id, inFinish ); + break; + case GenInlineItem::Ret: + RET( ret, inFinish ); + break; + case GenInlineItem::PChar: + ret << P(); + break; + case GenInlineItem::Char: + ret << GET_KEY(); + break; + case GenInlineItem::Hold: + ret << P() << " = " << P() << " - 1;"; + break; + case GenInlineItem::Exec: + EXEC( ret, item, targState, inFinish ); + break; + case GenInlineItem::Curs: + ret << "(_ps)"; + break; + case GenInlineItem::Targs: + ret << "(" << vCS() << ")"; + break; + case GenInlineItem::Entry: + ret << item->targState->id; + break; + case GenInlineItem::GotoExpr: + GOTO_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::CallExpr: + CALL_EXPR( ret, item, targState, inFinish ); + break; + case GenInlineItem::NextExpr: + NEXT_EXPR( ret, item, inFinish ); + break; + case GenInlineItem::LmSwitch: + LM_SWITCH( ret, item, targState, inFinish ); + break; + case GenInlineItem::LmSetActId: + SET_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokEnd: + SET_TOKEND( ret, item ); + break; + case GenInlineItem::LmGetTokEnd: + GET_TOKEND( ret, item ); + break; + case GenInlineItem::LmInitTokStart: + INIT_TOKSTART( ret, item ); + break; + case GenInlineItem::LmInitAct: + INIT_ACT( ret, item ); + break; + case GenInlineItem::LmSetTokStart: + SET_TOKSTART( ret, item ); + break; + case GenInlineItem::SubAction: + SUB_ACTION( ret, item, targState, inFinish ); + break; + case GenInlineItem::Break: + BREAK( ret, targState ); + break; + } + } +} + + +void RubyCodeGen::EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ) +{ + /* The parser gives fexec two children. The double brackets are for D + * code. If the inline list is a single word it will get interpreted as a + * C-style cast by the D compiler. */ + ret << " begin " << P() << " = (("; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << "))-1; end\n"; +} + +void RubyCodeGen::LM_SWITCH( ostream &ret, GenInlineItem *item, + int targState, int inFinish ) +{ + ret << + " case " << ACT() << "\n"; + + for ( GenInlineList::Iter lma = *item->children; lma.lte(); lma++ ) { + /* Write the case label, the action and the case break. */ + if ( lma->lmId < 0 ) + ret << " else\n"; + else + ret << " when " << lma->lmId << " then\n"; + + + /* Write the block and close it off. */ + ret << " begin"; + INLINE_LIST( ret, lma->children, targState, inFinish ); + ret << "end\n"; + } + + ret << "end \n\t"; +} + +void RubyCodeGen::SET_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = " << item->lmId << ";"; +} + +void RubyCodeGen::INIT_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << NULL_ITEM() << ";"; +} + +void RubyCodeGen::INIT_ACT( ostream &ret, GenInlineItem *item ) +{ + ret << ACT() << " = 0\n"; +} + +void RubyCodeGen::SET_TOKSTART( ostream &ret, GenInlineItem *item ) +{ + ret << TOKSTART() << " = " << P() << "\n"; +} + +void RubyCodeGen::SET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + /* The tokend action sets tokend. */ + ret << TOKEND() << " = " << P(); + if ( item->offset != 0 ) + out << "+" << item->offset; + out << "\n"; +} + +void RubyCodeGen::GET_TOKEND( ostream &ret, GenInlineItem *item ) +{ + ret << TOKEND(); +} + +void RubyCodeGen::SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ) +{ + if ( item->children->length() > 0 ) { + /* Write the block and close it off. */ + ret << " begin "; + INLINE_LIST( ret, item->children, targState, inFinish ); + ret << " end\n"; + } +} + +int RubyCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + return act; +} + +ostream &RubyCodeGen::source_warning( const InputLoc &loc ) +{ + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": warning: "; + return cerr; +} + +ostream &RubyCodeGen::source_error( const InputLoc &loc ) +{ + gblErrorCount += 1; + assert( sourceFileName != 0 ); + cerr << sourceFileName << ":" << loc.line << ":" << loc.col << ": "; + return cerr; +} + +void RubyCodeGen::finishRagelDef() +{ + if ( codeStyle == GenGoto || codeStyle == GenFGoto || + codeStyle == GenIpGoto || codeStyle == GenSplit ) + { + /* For directly executable machines there is no required state + * ordering. Choose a depth-first ordering to increase the + * potential for fall-throughs. */ + redFsm->depthFirstOrdering(); + } + else { + /* The frontend will do this for us, but it may be a good idea to + * force it if the intermediate file is edited. */ + redFsm->sortByStateId(); + } + + /* Choose default transitions and the single transition. */ + redFsm->chooseDefaultSpan(); + + /* Maybe do flat expand, otherwise choose single. */ + if ( codeStyle == GenFlat || codeStyle == GenFFlat ) + redFsm->makeFlat(); + else + redFsm->chooseSingle(); + + /* If any errors have occured in the input file then don't write anything. */ + if ( gblErrorCount > 0 ) + return; + + if ( codeStyle == GenSplit ) + redFsm->partitionFsm( numSplitPartitions ); + + if ( codeStyle == GenIpGoto || codeStyle == GenSplit ) + redFsm->setInTrans(); + + /* Anlayze Machine will find the final action reference counts, among + * other things. We will use these in reporting the usage + * of fsm directives in action code. */ + analyzeMachine(); + + /* Determine if we should use indicies. */ + calcIndexSize(); +} + + +/* Determine if we should use indicies or not. */ +void RubyCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActionLoc) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActionLoc) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +unsigned int RubyCodeGen::arrayTypeSize( unsigned long maxVal ) +{ + long long maxValLL = (long long) maxVal; + HostType *arrayType = keyOps->typeSubsumes( maxValLL ); + assert( arrayType != 0 ); + return arrayType->size; +} + + +void RubyCodeGen::writeInit() +{ + out << "begin\n"; + + out << " " << P() << " ||= 0\n"; + + if ( !noEnd ) + out << " " << PE() << " ||= " << DATA() << ".length\n"; + + if ( !noCS ) + out << " " << vCS() << " = " << START() << "\n"; + + /* If there are any calls, then the stack top needs initialization. */ + if ( redFsm->anyActionCalls() || redFsm->anyActionRets() ) + out << " " << TOP() << " = 0\n"; + + if ( hasLongestMatch ) { + out << + " " << TOKSTART() << " = " << NULL_ITEM() << "\n" + " " << TOKEND() << " = " << NULL_ITEM() << "\n" + " " << ACT() << " = 0\n"; + } + + out << "end\n"; +} + +void RubyCodeGen::writeExports() +{ + if ( exportList.length() > 0 ) { + for ( ExportList::Iter ex = exportList; ex.lte(); ex++ ) { + STATIC_VAR( ALPH_TYPE(), DATA_PREFIX() + "ex_" + ex->name ) + << " = " << KEY(ex->key) << "\n"; + } + out << "\n"; + } +} + +void RubyCodeGen::writeStart() +{ + out << START_STATE_ID(); +} + +void RubyCodeGen::writeFirstFinal() +{ + out << FIRST_FINAL_STATE(); +} + +void RubyCodeGen::writeError() +{ + out << ERROR_STATE(); +} + + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/contrib/tools/ragel6/rubycodegen.h b/contrib/tools/ragel6/rubycodegen.h new file mode 100644 index 0000000000..4e3a30d69a --- /dev/null +++ b/contrib/tools/ragel6/rubycodegen.h @@ -0,0 +1,174 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RUBY_CODEGEN_H +#define _RUBY_CODEGEN_H + +#include "common.h" +#include "gendata.h" + +/* Integer array line length. */ +#define IALL 8 + + +class RubyCodeGen : public CodeGenData +{ +public: + RubyCodeGen( ostream &out ) : CodeGenData(out) { } + virtual ~RubyCodeGen() {} +protected: + ostream &START_ARRAY_LINE(); + ostream &ARRAY_ITEM( string item, int count, bool last ); + ostream &END_ARRAY_LINE(); + + + string FSM_NAME(); + + string START_STATE_ID(); + string ERROR_STATE(); + string FIRST_FINAL_STATE(); + void INLINE_LIST(ostream &ret, GenInlineList *inlineList, int targState, bool inFinish); + string ACCESS(); + + void ACTION( ostream &ret, GenAction *action, int targState, bool inFinish ); + string GET_KEY(); + string GET_WIDE_KEY(); + string GET_WIDE_KEY( RedStateAp *state ); + string KEY( Key key ); + string TABS( int level ); + string INT( int i ); + void CONDITION( ostream &ret, GenAction *condition ); + string ALPH_TYPE(); + string WIDE_ALPH_TYPE(); + string ARRAY_TYPE( unsigned long maxVal ); + ostream &ACTIONS_ARRAY(); + void STATE_IDS(); + + + string DATA_PREFIX(); + string PM() { return "_" + DATA_PREFIX() + "partition_map"; } + string C() { return "_" + DATA_PREFIX() + "cond_spaces"; } + string CK() { return "_" + DATA_PREFIX() + "cond_keys"; } + string K() { return "_" + DATA_PREFIX() + "trans_keys"; } + string I() { return "_" + DATA_PREFIX() + "indicies"; } + string CO() { return "_" + DATA_PREFIX() + "cond_offsets"; } + string KO() { return "_" + DATA_PREFIX() + "key_offsets"; } + string IO() { return "_" + DATA_PREFIX() + "index_offsets"; } + string CL() { return "_" + DATA_PREFIX() + "cond_lengths"; } + string SL() { return "_" + DATA_PREFIX() + "single_lengths"; } + string RL() { return "_" + DATA_PREFIX() + "range_lengths"; } + string A() { return "_" + DATA_PREFIX() + "actions"; } + string TA() { return "_" + DATA_PREFIX() + "trans_actions"; } + string TT() { return "_" + DATA_PREFIX() + "trans_targs"; } + string TSA() { return "_" + DATA_PREFIX() + "to_state_actions"; } + string FSA() { return "_" + DATA_PREFIX() + "from_state_actions"; } + string EA() { return "_" + DATA_PREFIX() + "eof_actions"; } + string ET() { return "_" + DATA_PREFIX() + "eof_trans"; } + string SP() { return "_" + DATA_PREFIX() + "key_spans"; } + string CSP() { return "_" + DATA_PREFIX() + "cond_key_spans"; } + string START() { return DATA_PREFIX() + "start"; } + string ERROR() { return DATA_PREFIX() + "error"; } + string FIRST_FINAL() { return DATA_PREFIX() + "first_final"; } + string CTXDATA() { return DATA_PREFIX() + "ctxdata"; } + +public: + string NULL_ITEM(); + ostream &OPEN_ARRAY( string type, string name ); + ostream &CLOSE_ARRAY(); + ostream &STATIC_VAR( string type, string name ); + string ARR_OFF( string ptr, string offset ); + + string P(); + string PE(); + string vEOF(); + + string vCS(); + string TOP(); + string STACK(); + string ACT(); + string TOKSTART(); + string TOKEND(); + string DATA(); + + + void finishRagelDef(); + unsigned int arrayTypeSize( unsigned long maxVal ); + +protected: + virtual void writeExports(); + virtual void writeInit(); + virtual void writeStart(); + virtual void writeFirstFinal(); + virtual void writeError(); + + /* Determine if we should use indicies. */ + virtual void calcIndexSize(); + + virtual void BREAK( ostream &ret, int targState ) = 0; + virtual void GOTO( ostream &ret, int gotoDest, bool inFinish ) = 0; + virtual void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + virtual void CALL( ostream &ret, int callDest, int targState, bool inFinish ) = 0; + virtual void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ) = 0; + virtual void RET( ostream &ret, bool inFinish ) = 0; + + + virtual void NEXT( ostream &ret, int nextDest, bool inFinish ) = 0; + virtual void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) = 0; + + virtual int TO_STATE_ACTION( RedStateAp *state ) = 0; + virtual int FROM_STATE_ACTION( RedStateAp *state ) = 0; + virtual int EOF_ACTION( RedStateAp *state ) = 0; + + virtual int TRANS_ACTION( RedTransAp *trans ); + + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + void SUB_ACTION( ostream &ret, GenInlineItem *item, int targState, bool inFinish ); + +protected: + ostream &source_warning(const InputLoc &loc); + ostream &source_error(const InputLoc &loc); + + + /* fields */ + bool outLabelUsed; + bool againLabelUsed; + bool useIndicies; + + void genLineDirective( ostream &out ); +}; + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ + +#endif diff --git a/contrib/tools/ragel6/rubyfflat.cpp b/contrib/tools/ragel6/rubyfflat.cpp new file mode 100644 index 0000000000..53c7896c31 --- /dev/null +++ b/contrib/tools/ragel6/rubyfflat.cpp @@ -0,0 +1,488 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "rubyfflat.h" + +void RubyFFlatCodeGen::GOTO( ostream &out, int gotoDest, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = " << gotoDest << "\n" + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFFlatCodeGen::GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, 0, inFinish ); + out << ")\n"; + out << + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFFlatCodeGen::CALL( ostream &out, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << "+= 1\n" + " " << vCS() << " = " << callDest << "\n" + " _goto_level = _again\n" + " next\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFFlatCodeGen::CALL_EXPR(ostream &out, GenInlineItem *ilItem, + int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << " += 1\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, targState, inFinish ); + out << ")\n"; + + out << + " _goto_level = _again\n" + " next\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFFlatCodeGen::RET( ostream &out, bool inFinish ) +{ + out << + " begin\n" + " " << TOP() << " -= 1\n" + " " << vCS() << " = " << STACK() << "[" << TOP() << "]\n"; + + if ( postPopExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, postPopExpr, 0, false ); + out << "end\n"; + } + + out << + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFFlatCodeGen::BREAK( ostream &out, int targState ) +{ + out << + " begin\n" + " " << P() << " += 1\n" + " _goto_level = _out\n" + " next\n" + " end\n"; +} + + +int RubyFFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + return act; +} + +int RubyFFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + return act; +} + +int RubyFFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + return act; +} + +/* Write out the function for a transition. */ +int RubyFFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + return action; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &RubyFFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &RubyFFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyFFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &RubyFFlatCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + } + } + + genLineDirective( out ); + return out; +} + +void RubyFFlatCodeGen::writeData() +{ + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void RubyFFlatCodeGen::writeExec() +{ + out << + "begin\n" + " testEof = false\n" + " _slen, _trans, _keys, _inds"; + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + if ( redFsm->anyConditions() ) + out << ", _cond, _conds, _widec"; + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + out << ", _acts, _nacts"; + + out << " = nil\n"; + + out << + " _goto_level = 0\n" + " _resume = 10\n" + " _eof_trans = 15\n" + " _again = 20\n" + " _test_eof = 30\n" + " _out = 40\n"; + + out << + " while true\n" + " if _goto_level <= 0\n"; + + if ( !noEnd ) { + out << + " if " << P() << " == " << PE() << "\n" + " _goto_level = _test_eof\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + /* The resume label. */ + out << + " end\n" + " if _goto_level <= _resume\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " case " << FSA() << "[" << vCS() << "] \n"; + FROM_STATE_ACTION_SWITCH() << + " end\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) { + out << + " end\n" + " if _goto_level <= _eof_trans\n"; + } + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << "\n"; + + out << " " << vCS() << " = " << TT() << "[_trans]\n"; + + if ( redFsm->anyRegActions() ) { + /* break _again */ + out << + " if " << TA() << "[_trans] != 0\n" + " case " << TA() << "[_trans]" << "\n"; + ACTION_SWITCH() << + " end\n" + " end\n"; + } + + /* The again label. */ + out << + " end\n" + " if _goto_level <= _again\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " case " << TSA() << "[" << vCS() << "] \n"; + TO_STATE_ACTION_SWITCH() << + " end\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + out << " " << P() << " += 1\n"; + + if ( !noEnd ) { + out << + " if " << P() << " != " << PE() << "\n" + " _goto_level = _resume\n" + " next\n" + " end\n"; + } + else { + out << + " _goto_level = _resume\n" + " next\n"; + } + + /* The test eof label. */ + out << + " end\n" + " if _goto_level <= _test_eof\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << "\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " _goto_level = _eof_trans\n" + " next;\n" + " end\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " case " << EA() << "[" << vCS() << "]\n"; + EOF_ACTION_SWITCH() << + " end\n"; + } + + out << + " end\n" + "\n"; + } + + out << + " end\n" + " if _goto_level <= _out\n" + " break\n" + " end\n" + "end\n"; + + /* Wrapping the execute block. */ + out << " end\n"; +} + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ + diff --git a/contrib/tools/ragel6/rubyfflat.h b/contrib/tools/ragel6/rubyfflat.h new file mode 100644 index 0000000000..4ac412f9f6 --- /dev/null +++ b/contrib/tools/ragel6/rubyfflat.h @@ -0,0 +1,65 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RUBY_FFLATCODEGEN_H +#define _RUBY_FFLATCODEGEN_H + +#include <iostream> +#include "rubyflat.h" + +class RubyFFlatCodeGen : public RubyFlatCodeGen +{ +public: + RubyFFlatCodeGen( ostream &out ) : + RubyFlatCodeGen(out) {} +protected: + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + void GOTO( ostream &out, int gotoDest, bool inFinish ); + void GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ); + void CALL( ostream &out, int callDest, int targState, bool inFinish ); + void CALL_EXPR(ostream &out, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &out, bool inFinish ); + void BREAK( ostream &out, int targState ); + + virtual int TO_STATE_ACTION( RedStateAp *state ); + virtual int FROM_STATE_ACTION( RedStateAp *state ); + virtual int EOF_ACTION( RedStateAp *state ); + virtual int TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); +}; + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ + +#endif diff --git a/contrib/tools/ragel6/rubyflat.cpp b/contrib/tools/ragel6/rubyflat.cpp new file mode 100644 index 0000000000..827084d2c2 --- /dev/null +++ b/contrib/tools/ragel6/rubyflat.cpp @@ -0,0 +1,873 @@ +/* + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + * Copyright 2007 Victor Hugo Borja <vic@rubyforge.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include "rubyflat.h" +#include "ragel.h" +#include "redfsm.h" +#include "gendata.h" + +using std::ostream; +using std::string; + +std::ostream &RubyFlatCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyFlatCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyFlatCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, true ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyFlatCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break */ + out << "\twhen " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &RubyFlatCodeGen::KEYS() +{ + START_ARRAY_LINE(); + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just low key and high key. */ + ARRAY_ITEM( KEY( st->lowKey ), ++totalTrans, false ); + ARRAY_ITEM( KEY( st->highKey ), ++totalTrans, false ); + if ( ++totalTrans % IALL == 0 ) + out << "\n\t"; + + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT( 0 ), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::INDICIES() +{ + int totalTrans = 0; + START_ARRAY_LINE(); + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->transList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->lowKey, st->highKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + ARRAY_ITEM( KEY( st->transList[pos]->id ), ++totalTrans, false ); + } + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) + ARRAY_ITEM( KEY( st->defTrans->id ), ++totalTrans, false ); + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT( 0 ), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::FLAT_INDEX_OFFSET() +{ + START_ARRAY_LINE(); + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + ARRAY_ITEM( INT( curIndOffset ), ++totalStateNum, st.last() ); + /* Move the index offset ahead. */ + if ( st->transList != 0 ) + curIndOffset += keyOps->span( st->lowKey, st->highKey ); + + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::KEY_SPANS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->transList != 0 ) + span = keyOps->span( st->lowKey, st->highKey ); + ARRAY_ITEM( INT( span ), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::TO_STATE_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT( TO_STATE_ACTION(st) ), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::FROM_STATE_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT( FROM_STATE_ACTION(st) ), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::EOF_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT( EOF_ACTION(st) ), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::EOF_TRANS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + + /* Write any eof action. */ + ARRAY_ITEM( INT(trans), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::TRANS_TARGS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + START_ARRAY_LINE(); + + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Save the position. Needed for eofTargs. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + ARRAY_ITEM( INT( trans->targ->id ), ++totalStates, t >= redFsm->transSet.length()-1 ); + } + END_ARRAY_LINE(); + delete[] transPtrs; + return out; +} + + +std::ostream &RubyFlatCodeGen::TRANS_ACTIONS() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + START_ARRAY_LINE(); + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + ARRAY_ITEM( INT( TRANS_ACTION( trans ) ), ++totalAct, t >= redFsm->transSet.length()-1 ); + } + END_ARRAY_LINE(); + delete[] transPtrs; + return out; +} + + +void RubyFlatCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << vCS() << " << 1\n" + " _inds = " << IO() << "[" << vCS() << "]\n" + " _slen = " << SP() << "[" << vCS() << "]\n" + " _wide = " << GET_WIDE_KEY() << "\n" + " _trans = if ( _slen > 0 && \n" + " " << K() << "[_keys] <= _wide && \n" + " " << "_wide <= " << K() << "[_keys + 1] \n" + " ) then\n" + " " << I() << "[ _inds + _wide - " << K() << "[_keys] ] \n" + " else \n" + " " << I() << "[ _inds + _slen ]\n" + " end\n" + ""; + +} + +std::ostream &RubyFlatCodeGen::COND_INDEX_OFFSET() +{ + START_ARRAY_LINE(); + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + ARRAY_ITEM( INT( curIndOffset ), ++totalStateNum, st.last() ); + /* Move the index offset ahead. */ + if ( st->condList != 0 ) + curIndOffset += keyOps->span( st->condLowKey, st->condHighKey ); + } + END_ARRAY_LINE(); + return out; +} + +void RubyFlatCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << "\n" + " _keys = " << vCS() << " << 1\n" + " _conds = " << CO() << "[" << vCS() << "]\n" + " _slen = " << CSP() << "[" << vCS() << "]\n" + " _wide = " << GET_WIDE_KEY() << "\n" + " _cond = if ( _slen > 0 && \n" + " " << CK() << "[_keys] <= _wide &&\n" + " " << "_wide <= " << CK() << "[_keys + 1]\n" + " ) then \n" + " " << C() << "[ _conds + _wide - " << CK() << "[_keys]" << " ]\n" + " else\n" + " 0\n" + " end\n"; + out << + " case _cond \n"; + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " when " << condSpace->condSpaceId + 1 << " then\n"; + out << TABS(2) << "_widec = " << "(" << + KEY(condSpace->baseKey) << " + (" << GET_KEY() << + " - " << KEY(keyOps->minKey) << "))\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + out << TABS(2) << "if ( "; + CONDITION( out, *csi ); + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << + " ) then \n" << + TABS(3) << " _widec += " << condValOffset << "\n" + "end\n"; + } + } + + out << + " end # _cond switch \n"; +} + +std::ostream &RubyFlatCodeGen::CONDS() +{ + int totalTrans = 0; + START_ARRAY_LINE(); + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->condList != 0 ) { + /* Walk the singles. */ + unsigned long long span = keyOps->span( st->condLowKey, st->condHighKey ); + for ( unsigned long long pos = 0; pos < span; pos++ ) { + if ( st->condList[pos] != 0 ) + ARRAY_ITEM( INT( st->condList[pos]->condSpaceId + 1 ), ++totalTrans, false ); + else + ARRAY_ITEM( INT( 0 ), ++totalTrans, false ); + } + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT( 0 ), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::COND_KEYS() +{ + START_ARRAY_LINE(); + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit just cond low key and cond high key. */ + ARRAY_ITEM( KEY( st->condLowKey ), ++totalTrans, false ); + ARRAY_ITEM( KEY( st->condHighKey ), ++totalTrans, false ); + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT( 0 ), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyFlatCodeGen::COND_KEY_SPANS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + unsigned long long span = 0; + if ( st->condList != 0 ) + span = keyOps->span( st->condLowKey, st->condHighKey ); + ARRAY_ITEM( INT( span ), ++totalStateNum, false ); + } + END_ARRAY_LINE(); + return out; +} + + +void RubyFlatCodeGen::GOTO( ostream &out, int gotoDest, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = " << gotoDest << "\n" + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyFlatCodeGen::CALL( ostream &out, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << "+= 1\n" + " " << vCS() << " = " << callDest << "\n" + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFlatCodeGen::CALL_EXPR(ostream &out, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << " += 1\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, targState, inFinish ); + out << ")\n"; + + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFlatCodeGen::RET( ostream &out, bool inFinish ) +{ + out << + " begin\n" + " " << TOP() << " -= 1\n" + " " << vCS() << " = " << STACK() << "[" << TOP() << "]\n"; + + if ( postPopExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, postPopExpr, 0, false ); + out << "end\n"; + } + + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyFlatCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void RubyFlatCodeGen::GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, 0, inFinish ); + out << ")\n"; + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyFlatCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + + +void RubyFlatCodeGen::CURS( ostream &ret, bool inFinish ) +{ + ret << "(_ps)"; +} + +void RubyFlatCodeGen::TARGS( ostream &ret, bool inFinish, int targState ) +{ + ret << "(" << vCS() << ")"; +} + +void RubyFlatCodeGen::BREAK( ostream &out, int targState ) +{ + out << + " begin\n" + " " << P() << " += 1\n" + " _trigger_goto = true\n" + " _goto_level = _out\n" + " break\n" + " end\n"; +} + +int RubyFlatCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +int RubyFlatCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +int RubyFlatCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + +int RubyFlatCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + /* If there are actions, emit them. Otherwise emit zero. */ + int act = 0; + if ( trans->action != 0 ) + act = trans->action->location+1; + return act; +} + +void RubyFlatCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpan), CSP() ); + COND_KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCond), C() ); + CONDS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondIndexOffset), CO() ); + COND_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSpan), SP() ); + KEY_SPANS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxFlatIndexOffset), IO() ); + FLAT_INDEX_OFFSET(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void RubyFlatCodeGen::writeExec() +{ + out << + "begin # ragel flat\n" + " testEof = false\n" + " _slen, _trans, _keys, _inds"; + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + if ( redFsm->anyConditions() ) + out << ", _cond, _conds, _widec"; + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + out << ", _acts, _nacts"; + + out << " = nil\n"; + + out << + " _goto_level = 0\n" + " _resume = 10\n" + " _eof_trans = 15\n" + " _again = 20\n" + " _test_eof = 30\n" + " _out = 40\n"; + + out << + " while true\n" + " _trigger_goto = false\n" + " if _goto_level <= 0\n"; + + if ( !noEnd ) { + out << + " if " << P() << " == " << PE() << "\n" + " _goto_level = _test_eof\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + /* The resume label. */ + out << + " end\n" + " if _goto_level <= _resume\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" << vCS() << "]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + FROM_STATE_ACTION_SWITCH(); + out << + " end # from state action switch\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( redFsm->anyEofTrans() ) { + out << + " end\n" + " if _goto_level <= _eof_trans\n"; + } + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << "\n"; + + out << " " << vCS() << " = " << TT() << "[_trans]\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if " << TA() << "[_trans] != 0\n" + " _acts = " << TA() << "[_trans]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + ACTION_SWITCH(); + out << + " end # action switch\n" + " end\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + /* The again label. */ + out << + " end\n" + " if _goto_level <= _again\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + TO_STATE_ACTION_SWITCH() << + " end # to state action switch\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + out << " " << P() << " += 1\n"; + + if ( !noEnd ) { + out << + " if " << P() << " != " << PE() << "\n" + " _goto_level = _resume\n" + " next\n" + " end\n"; + } + else { + out << + " _goto_level = _resume\n" + " next\n"; + } + + /* The test_eof label. */ + out << + " end\n" + " if _goto_level <= _test_eof\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << "\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " _goto_level = _eof_trans\n" + " next;\n" + " end\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " begin\n" + " __acts = " << EA() << "[" << vCS() << "]\n" + " __nacts = " << A() << "[__acts]\n" << + " __acts += 1\n" + " while ( __nacts > 0 ) \n" + " __nacts -= 1\n" + " __acts += 1\n" + " case ( "<< A() << "[__acts-1] ) \n"; + EOF_ACTION_SWITCH() << + " end\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n" + " end\n"; + } + + out << + " end\n"; + } + + out << + " end\n" + " if _goto_level <= _out\n" + " break\n" + " end\n"; + + /* The loop for faking goto. */ + out << + " end\n"; + + /* Wrapping the execute block. */ + out << + " end\n"; +} + + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/contrib/tools/ragel6/rubyflat.h b/contrib/tools/ragel6/rubyflat.h new file mode 100644 index 0000000000..5136791069 --- /dev/null +++ b/contrib/tools/ragel6/rubyflat.h @@ -0,0 +1,99 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RUBY_FLATCODEGEN_H +#define _RUBY_FLATCODEGEN_H + +#include <iostream> +#include "rubycodegen.h" + +using std::string; +using std::ostream; + + +/* + * FlatCodeGen + */ +class RubyFlatCodeGen : public RubyCodeGen +{ +public: + RubyFlatCodeGen( ostream &out ) : + RubyCodeGen(out) {}; + virtual ~RubyFlatCodeGen() {} +protected: + + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &FLAT_INDEX_OFFSET(); + std::ostream &KEY_SPANS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + void LOCATE_TRANS(); + + std::ostream &COND_INDEX_OFFSET(); + void COND_TRANSLATE(); + std::ostream &CONDS(); + std::ostream &COND_KEYS(); + std::ostream &COND_KEY_SPANS(); + + + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void CURS( ostream &ret, bool inFinish ); + void TARGS( ostream &ret, bool inFinish, int targState ); + void RET( ostream &ret, bool inFinish ); + void BREAK( ostream &ret, int targState ); + + + virtual int TO_STATE_ACTION( RedStateAp *state ); + virtual int FROM_STATE_ACTION( RedStateAp *state ); + virtual int EOF_ACTION( RedStateAp *state ); + virtual int TRANS_ACTION( RedTransAp *trans ); + + virtual void writeData(); + virtual void writeExec(); + +}; + +#endif + + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/contrib/tools/ragel6/rubyftable.cpp b/contrib/tools/ragel6/rubyftable.cpp new file mode 100644 index 0000000000..b90ffa7b4c --- /dev/null +++ b/contrib/tools/ragel6/rubyftable.cpp @@ -0,0 +1,563 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iomanip> +#include <sstream> +#include "redfsm.h" +#include "gendata.h" +#include "ragel.h" +#include "rubyftable.h" + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; + +void RubyFTabCodeGen::GOTO( ostream &out, int gotoDest, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = " << gotoDest << "\n" + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFTabCodeGen::GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, 0, inFinish ); + out << ")\n"; + out << + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFTabCodeGen::CALL( ostream &out, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << "+= 1\n" + " " << vCS() << " = " << callDest << "\n" + " _goto_level = _again\n" + " next\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFTabCodeGen::CALL_EXPR(ostream &out, GenInlineItem *ilItem, + int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << " += 1\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, targState, inFinish ); + out << ")\n"; + + out << + " _goto_level = _again\n" + " next\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyFTabCodeGen::RET( ostream &out, bool inFinish ) +{ + out << + " begin\n" + " " << TOP() << " -= 1\n" + " " << vCS() << " = " << STACK() << "[" << TOP() << "]\n"; + + if ( postPopExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, postPopExpr, 0, false ); + out << "end\n"; + } + + out << + " _goto_level = _again\n" + " next\n" + " end\n"; +} + +void RubyFTabCodeGen::BREAK( ostream &out, int targState ) +{ + out << + " begin\n" + " " << P() << " += 1\n" + " _goto_level = _out\n" + " next\n" + " end\n"; +} + + +std::ostream &RubyFTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numToStateRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &RubyFTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numFromStateRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyFTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numEofRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, true ); + + } + } + + genLineDirective( out ); + return out; +} + +/* Write out the function switch. This switch is keyed on the values + * of the func index. */ +std::ostream &RubyFTabCodeGen::ACTION_SWITCH() +{ + /* Loop the actions. */ + for ( GenActionTableMap::Iter redAct = redFsm->actionMap; redAct.lte(); redAct++ ) { + if ( redAct->numTransRefs > 0 ) { + /* Write the entry label. */ + out << "\twhen " << redAct->actListId+1 << " then\n"; + + /* Write each action in the list of action items. */ + for ( GenActionTable::Iter item = redAct->key; item.lte(); item++ ) + ACTION( out, item->value, 0, false ); + + } + } + + genLineDirective( out ); + return out; +} + + +int RubyFTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->actListId+1; + return act; +} + +int RubyFTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->actListId+1; + return act; +} + +int RubyFTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->actListId+1; + return act; +} + + +/* Write out the function for a transition. */ +int RubyFTabCodeGen::TRANS_ACTION( RedTransAp *trans ) +{ + int action = 0; + if ( trans->action != 0 ) + action = trans->action->actListId+1; + return action; +} + +void RubyFTabCodeGen::writeData() +{ + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActListId), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +void RubyFTabCodeGen::writeExec() +{ + out << + "begin\n" + " testEof = false\n" + " _klen, _trans, _keys"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + + if ( redFsm->anyConditions() ) + out << ", _widec"; + + out << " = nil\n"; + + out << + " _goto_level = 0\n" + " _resume = 10\n" + " _eof_trans = 15\n" + " _again = 20\n" + " _test_eof = 30\n" + " _out = 40\n"; + + out << + " while true\n" + " if _goto_level <= 0\n"; + + if ( !noEnd ) { + out << + " if " << P() << " == " << PE() << "\n" + " _goto_level = _test_eof\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + /* The resume label. */ + out << + " end\n" + " if _goto_level <= _resume\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " case " << FSA() << "[" << vCS() << "] \n"; + FROM_STATE_ACTION_SWITCH() << + " end # from state action switch \n" + "\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( useIndicies ) + out << " _trans = " << I() << "[_trans];\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " end\n" + " if _goto_level <= _eof_trans\n"; + } + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << ";\n"; + + out << + " " << vCS() << " = " << TT() << "[_trans];\n" + "\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if " << TA() << "[_trans] != 0\n" + "\n" + " case " << TA() << "[_trans] \n"; + ACTION_SWITCH() << + " end # action switch \n" + " end\n" + "\n"; + } + + /* The again label. */ + out << + " end\n" + " if _goto_level <= _again\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " case " << TSA() << "[" << vCS() << "] \n"; + TO_STATE_ACTION_SWITCH() << + " end\n" + "\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + out << " " << P() << " += 1\n"; + + if ( !noEnd ) { + out << + " if " << P() << " != " << PE() << "\n" + " _goto_level = _resume\n" + " next\n" + " end\n"; + } + else { + out << + " _goto_level = _resume\n" + " next\n"; + } + + /* The test eof label. */ + out << + " end\n" + " if _goto_level <= _test_eof\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << "\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " _goto_level = _eof_trans\n" + " next;\n" + " end\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " begin\n" + " case ( " << EA() << "[" << vCS() << "] )\n"; + EOF_ACTION_SWITCH() << + " end\n" + " end\n"; + } + + out << + " end\n" + "\n"; + } + + out << + " end\n" + " if _goto_level <= _out\n" + " break\n" + " end\n" + "end\n"; + + /* Wrapping the execute block. */ + out << " end\n"; +} + + +void RubyFTabCodeGen::calcIndexSize() +{ + int sizeWithInds = 0, sizeWithoutInds = 0; + + /* Calculate cost of using with indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithInds += arrayTypeSize(redFsm->maxIndex) * totalIndex; + } + sizeWithInds += arrayTypeSize(redFsm->maxState) * redFsm->transSet.length(); + if ( redFsm->anyActions() ) + sizeWithInds += arrayTypeSize(redFsm->maxActListId) * redFsm->transSet.length(); + + /* Calculate the cost of not using indicies. */ + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + int totalIndex = st->outSingle.length() + st->outRange.length() + + (st->defTrans == 0 ? 0 : 1); + sizeWithoutInds += arrayTypeSize(redFsm->maxState) * totalIndex; + if ( redFsm->anyActions() ) + sizeWithoutInds += arrayTypeSize(redFsm->maxActListId) * totalIndex; + } + + /* If using indicies reduces the size, use them. */ + useIndicies = sizeWithInds < sizeWithoutInds; +} + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/contrib/tools/ragel6/rubyftable.h b/contrib/tools/ragel6/rubyftable.h new file mode 100644 index 0000000000..91d7fe501a --- /dev/null +++ b/contrib/tools/ragel6/rubyftable.h @@ -0,0 +1,64 @@ +/* + * 2007 Victor Hugo Borja <vic@rubyforge.org> + * Copyright 2001-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RUBY_FTABCODEGEN_H +#define _RUBY_FTABCODEGEN_H + +#include "rubytable.h" + +class RubyFTabCodeGen : public RubyTabCodeGen +{ +public: + RubyFTabCodeGen( ostream &out ): RubyTabCodeGen(out) {} +protected: + std::ostream &TO_STATE_ACTION_SWITCH(); + std::ostream &FROM_STATE_ACTION_SWITCH(); + std::ostream &EOF_ACTION_SWITCH(); + std::ostream &ACTION_SWITCH(); + + void GOTO( ostream &out, int gotoDest, bool inFinish ); + void GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ); + void CALL( ostream &out, int callDest, int targState, bool inFinish ); + void CALL_EXPR(ostream &out, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &out, bool inFinish ); + void BREAK( ostream &out, int targState ); + + int TO_STATE_ACTION( RedStateAp *state ); + int FROM_STATE_ACTION( RedStateAp *state ); + int EOF_ACTION( RedStateAp *state ); + virtual int TRANS_ACTION( RedTransAp *trans ); + + void writeData(); + void writeExec(); + void calcIndexSize(); +}; + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ + +#endif + diff --git a/contrib/tools/ragel6/rubytable.cpp b/contrib/tools/ragel6/rubytable.cpp new file mode 100644 index 0000000000..eb5dfd5c73 --- /dev/null +++ b/contrib/tools/ragel6/rubytable.cpp @@ -0,0 +1,1033 @@ +/* + * Copyright 2007 Victor Hugo Borja <vic@rubyforge.org> + * 2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#include <iomanip> +#include <sstream> +#include "redfsm.h" +#include "gendata.h" +#include "ragel.h" +#include "rubytable.h" + +using std::ostream; +using std::ostringstream; +using std::string; +using std::cerr; +using std::endl; + + + +void RubyTabCodeGen::GOTO( ostream &out, int gotoDest, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = " << gotoDest << "\n" + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyTabCodeGen::GOTO_EXPR( ostream &out, GenInlineItem *ilItem, bool inFinish ) +{ + out << + " begin\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, 0, inFinish ); + out << ")\n"; + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyTabCodeGen::CALL( ostream &out, int callDest, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << "+= 1\n" + " " << vCS() << " = " << callDest << "\n" + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyTabCodeGen::CALL_EXPR(ostream &out, GenInlineItem *ilItem, int targState, bool inFinish ) +{ + if ( prePushExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, prePushExpr, 0, false ); + } + + out << + " begin\n" + " " << STACK() << "[" << TOP() << "] = " << vCS() << "\n" + " " << TOP() << " += 1\n" + " " << vCS() << " = ("; + INLINE_LIST( out, ilItem->children, targState, inFinish ); + out << ")\n"; + + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; + + if ( prePushExpr != 0 ) + out << "end\n"; +} + +void RubyTabCodeGen::RET( ostream &out, bool inFinish ) +{ + out << + " begin\n" + " " << TOP() << " -= 1\n" + " " << vCS() << " = " << STACK() << "[" << TOP() << "]\n"; + + if ( postPopExpr != 0 ) { + out << "begin\n"; + INLINE_LIST( out, postPopExpr, 0, false ); + out << "end\n"; + } + + out << + " _trigger_goto = true\n" + " _goto_level = _again\n" + " break\n" + " end\n"; +} + +void RubyTabCodeGen::BREAK( ostream &out, int targState ) +{ + out << + " begin\n" + " " << P() << " += 1\n" + " _trigger_goto = true\n" + " _goto_level = _out\n" + " break\n" + " end\n"; +} + +void RubyTabCodeGen::COND_TRANSLATE() +{ + out << + " _widec = " << GET_KEY() << "\n" + " _keys = " << CO() << "[" << vCS() << "]*2\n" + " _klen = " << CL() << "[" << vCS() << "]\n" + " if _klen > 0\n" + " _lower = _keys\n" + " _upper = _keys + (_klen<<1) - 2\n" + " loop do\n" + " break if _upper < _lower\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1)\n" + " if " << GET_WIDE_KEY() << " < " << CK() << "[_mid]\n" + " _upper = _mid - 2\n" + " elsif " << GET_WIDE_KEY() << " > " << CK() << "[_mid+1]\n" + " _lower = _mid + 2\n" + " else\n" + " case " << C() << "[" << CO() << "[" << vCS() << "]" + " + ((_mid - _keys)>>1)]\n"; + + for ( CondSpaceList::Iter csi = condSpaceList; csi.lte(); csi++ ) { + GenCondSpace *condSpace = csi; + out << " when " << condSpace->condSpaceId << " then" ; + out << " _widec = " << KEY(condSpace->baseKey) << + "+ (" << GET_KEY() << " - " << KEY(keyOps->minKey) << ")\n"; + + for ( GenCondSet::Iter csi = condSpace->condSet; csi.lte(); csi++ ) { + Size condValOffset = ((1 << csi.pos()) * keyOps->alphSize()); + out << " _widec += " << condValOffset << " if ( "; + CONDITION( out, *csi ); + out << " )\n"; + } + } + + out << + " end # case\n" + " end\n" + " end # loop\n" + " end\n"; +} + + +void RubyTabCodeGen::LOCATE_TRANS() +{ + out << + " _keys = " << KO() << "[" << vCS() << "]\n" + " _trans = " << IO() << "[" << vCS() << "]\n" + " _klen = " << SL() << "[" << vCS() << "]\n" + " _break_match = false\n" + " \n" + " begin\n" + " if _klen > 0\n" + " _lower = _keys\n" + " _upper = _keys + _klen - 1\n" + "\n" + " loop do\n" + " break if _upper < _lower\n" + " _mid = _lower + ( (_upper - _lower) >> 1 )\n" + "\n" + " if " << GET_WIDE_KEY() << " < " << K() << "[_mid]\n" + " _upper = _mid - 1\n" + " elsif " << GET_WIDE_KEY() << " > " << K() << "[_mid]\n" + " _lower = _mid + 1\n" + " else\n" + " _trans += (_mid - _keys)\n" + " _break_match = true\n" + " break\n" + " end\n" + " end # loop\n" + " break if _break_match\n" + " _keys += _klen\n" + " _trans += _klen\n" + " end" + "\n" + " _klen = " << RL() << "[" << vCS() << "]\n" + " if _klen > 0\n" + " _lower = _keys\n" + " _upper = _keys + (_klen << 1) - 2\n" + " loop do\n" + " break if _upper < _lower\n" + " _mid = _lower + (((_upper-_lower) >> 1) & ~1)\n" + " if " << GET_WIDE_KEY() << " < " << K() << "[_mid]\n" + " _upper = _mid - 2\n" + " elsif " << GET_WIDE_KEY() << " > " << K() << "[_mid+1]\n" + " _lower = _mid + 2\n" + " else\n" + " _trans += ((_mid - _keys) >> 1)\n" + " _break_match = true\n" + " break\n" + " end\n" + " end # loop\n" + " break if _break_match\n" + " _trans += _klen\n" + " end\n" + " end while false\n"; +} + +void RubyTabCodeGen::writeExec() +{ + out << + "begin\n" + " _klen, _trans, _keys"; + + if ( redFsm->anyRegCurStateRef() ) + out << ", _ps"; + if ( redFsm->anyConditions() ) + out << ", _widec"; + if ( redFsm->anyToStateActions() || redFsm->anyRegActions() + || redFsm->anyFromStateActions() ) + out << ", _acts, _nacts"; + + out << " = nil\n"; + + out << + " _goto_level = 0\n" + " _resume = 10\n" + " _eof_trans = 15\n" + " _again = 20\n" + " _test_eof = 30\n" + " _out = 40\n"; + + out << + " while true\n" + " _trigger_goto = false\n" + " if _goto_level <= 0\n"; + + if ( !noEnd ) { + out << + " if " << P() << " == " << PE() << "\n" + " _goto_level = _test_eof\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + /* The resume label. */ + out << + " end\n" + " if _goto_level <= _resume\n"; + + if ( redFsm->anyFromStateActions() ) { + out << + " _acts = " << FSA() << "[" << vCS() << "]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + FROM_STATE_ACTION_SWITCH(); + out << + " end # from state action switch\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + if ( redFsm->anyConditions() ) + COND_TRANSLATE(); + + LOCATE_TRANS(); + + if ( useIndicies ) + out << " _trans = " << I() << "[_trans]\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " end\n" + " if _goto_level <= _eof_trans\n"; + } + + if ( redFsm->anyRegCurStateRef() ) + out << " _ps = " << vCS() << "\n"; + + out << " " << vCS() << " = " << TT() << "[_trans]\n"; + + if ( redFsm->anyRegActions() ) { + out << + " if " << TA() << "[_trans] != 0\n" + " _acts = " << TA() << "[_trans]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + ACTION_SWITCH(); + out << + " end # action switch\n" + " end\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + /* The again label. */ + out << + " end\n" + " if _goto_level <= _again\n"; + + if ( redFsm->anyToStateActions() ) { + out << + " _acts = " << TSA() << "[" << vCS() << "]\n" + " _nacts = " << A() << "[_acts]\n" + " _acts += 1\n" + " while _nacts > 0\n" + " _nacts -= 1\n" + " _acts += 1\n" + " case " << A() << "[_acts - 1]\n"; + TO_STATE_ACTION_SWITCH(); + out << + " end # to state action switch\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + if ( redFsm->errState != 0 ) { + out << + " if " << vCS() << " == " << redFsm->errState->id << "\n" + " _goto_level = _out\n" + " next\n" + " end\n"; + } + + out << " " << P() << " += 1\n"; + + if ( !noEnd ) { + out << + " if " << P() << " != " << PE() << "\n" + " _goto_level = _resume\n" + " next\n" + " end\n"; + } + else { + out << + " _goto_level = _resume\n" + " next\n"; + } + + /* The test_eof label. */ + out << + " end\n" + " if _goto_level <= _test_eof\n"; + + if ( redFsm->anyEofTrans() || redFsm->anyEofActions() ) { + out << + " if " << P() << " == " << vEOF() << "\n"; + + if ( redFsm->anyEofTrans() ) { + out << + " if " << ET() << "[" << vCS() << "] > 0\n" + " _trans = " << ET() << "[" << vCS() << "] - 1;\n" + " _goto_level = _eof_trans\n" + " next;\n" + " end\n"; + } + + if ( redFsm->anyEofActions() ) { + out << + " __acts = " << EA() << "[" << vCS() << "]\n" + " __nacts = " << " " << A() << "[__acts]\n" + " __acts += 1\n" + " while __nacts > 0\n" + " __nacts -= 1\n" + " __acts += 1\n" + " case " << A() << "[__acts - 1]\n"; + EOF_ACTION_SWITCH() << + " end # eof action switch\n" + " end\n" + " if _trigger_goto\n" + " next\n" + " end\n"; + } + + out << + "end\n"; + } + + out << + " end\n" + " if _goto_level <= _out\n" + " break\n" + " end\n"; + + /* The loop for next. */ + out << + " end\n"; + + /* Wrapping the execute block. */ + out << + " end\n"; +} + +std::ostream &RubyTabCodeGen::FROM_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numFromStateRefs > 0 ) { + /* Write the case label, the action */ + out << " when " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + + +std::ostream &RubyTabCodeGen::TO_STATE_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numToStateRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "when " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyTabCodeGen::EOF_ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numEofRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "when " << act->actionId << " then\n"; + ACTION( out, act, 0, true ); + } + } + + genLineDirective( out ); + return out; +} + +std::ostream &RubyTabCodeGen::ACTION_SWITCH() +{ + /* Walk the list of functions, printing the cases. */ + for ( GenActionList::Iter act = actionList; act.lte(); act++ ) { + /* Write out referenced actions. */ + if ( act->numTransRefs > 0 ) { + /* Write the case label, the action and the case break. */ + out << "when " << act->actionId << " then\n"; + ACTION( out, act, 0, false ); + } + } + + genLineDirective( out ); + return out; +} + + +void RubyTabCodeGen::NEXT( ostream &ret, int nextDest, bool inFinish ) +{ + ret << vCS() << " = " << nextDest << ";"; +} + +void RubyTabCodeGen::NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ) +{ + ret << vCS() << " = ("; + INLINE_LIST( ret, ilItem->children, 0, inFinish ); + ret << ");"; +} + + +int RubyTabCodeGen::TO_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->toStateAction != 0 ) + act = state->toStateAction->location+1; + return act; +} + +int RubyTabCodeGen::FROM_STATE_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->fromStateAction != 0 ) + act = state->fromStateAction->location+1; + return act; +} + +int RubyTabCodeGen::EOF_ACTION( RedStateAp *state ) +{ + int act = 0; + if ( state->eofAction != 0 ) + act = state->eofAction->location+1; + return act; +} + + +std::ostream &RubyTabCodeGen::COND_OFFSETS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + ARRAY_ITEM( INT(curKeyOffset), ++totalStateNum, st.last() ); + + /* Move the key offset ahead. */ + curKeyOffset += st->stateCondList.length(); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::KEY_OFFSETS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0, curKeyOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the key offset. */ + ARRAY_ITEM( INT(curKeyOffset), ++totalStateNum, st.last() ); + + /* Move the key offset ahead. */ + curKeyOffset += st->outSingle.length() + st->outRange.length()*2; + } + END_ARRAY_LINE(); + return out; +} + + +std::ostream &RubyTabCodeGen::INDEX_OFFSETS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0, curIndOffset = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write the index offset. */ + ARRAY_ITEM( INT(curIndOffset), ++totalStateNum, st.last() ); + + /* Move the index offset ahead. */ + curIndOffset += st->outSingle.length() + st->outRange.length(); + if ( st->defTrans != 0 ) + curIndOffset += 1; + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::COND_LENS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + ARRAY_ITEM( INT(st->stateCondList.length()), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + + +std::ostream &RubyTabCodeGen::SINGLE_LENS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write singles length. */ + ARRAY_ITEM( INT(st->outSingle.length()), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::RANGE_LENS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Emit length of range index. */ + ARRAY_ITEM( INT(st->outRange.length()), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::TO_STATE_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(TO_STATE_ACTION(st)), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::FROM_STATE_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(FROM_STATE_ACTION(st)), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::EOF_ACTIONS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + ARRAY_ITEM( INT(EOF_ACTION(st)), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::EOF_TRANS() +{ + START_ARRAY_LINE(); + int totalStateNum = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Write any eof action. */ + long trans = 0; + if ( st->eofTrans != 0 ) { + assert( st->eofTrans->pos >= 0 ); + trans = st->eofTrans->pos+1; + } + + /* Write any eof action. */ + ARRAY_ITEM( INT(trans), ++totalStateNum, st.last() ); + } + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::COND_KEYS() +{ + START_ARRAY_LINE(); + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Lower key. */ + ARRAY_ITEM( KEY( sc->lowKey ), ++totalTrans, false ); + ARRAY_ITEM( KEY( sc->highKey ), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::COND_SPACES() +{ + START_ARRAY_LINE(); + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the state's transitions. */ + for ( GenStateCondList::Iter sc = st->stateCondList; sc.lte(); sc++ ) { + /* Cond Space id. */ + ARRAY_ITEM( KEY( sc->condSpace->condSpaceId ), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::KEYS() +{ + START_ARRAY_LINE(); + int totalTrans = 0; + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Loop the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + ARRAY_ITEM( KEY( stel->lowKey ), ++totalTrans, false ); + } + + /* Loop the state's transitions. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + /* Lower key. */ + ARRAY_ITEM( KEY( rtel->lowKey ), ++totalTrans, false ); + + /* Upper key. */ + ARRAY_ITEM( KEY( rtel->highKey ), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::INDICIES() +{ + int totalTrans = 0; + START_ARRAY_LINE(); + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + ARRAY_ITEM( KEY( stel->value->id ), ++totalTrans, false ); + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + ARRAY_ITEM( KEY( rtel->value->id ), ++totalTrans, false ); + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + ARRAY_ITEM( KEY( st->defTrans->id ), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::TRANS_TARGS() +{ + int totalTrans = 0; + START_ARRAY_LINE(); + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + ARRAY_ITEM( KEY( trans->targ->id ), ++totalTrans, false ); + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + ARRAY_ITEM( KEY( trans->targ->id ), ++totalTrans, false ); + } + + /* The state's default target state. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + ARRAY_ITEM( KEY( trans->targ->id ), ++totalTrans, false ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + trans->pos = totalTrans; + ARRAY_ITEM( KEY( trans->targ->id ), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + + +std::ostream &RubyTabCodeGen::TRANS_ACTIONS() +{ + int totalTrans = 0; + START_ARRAY_LINE(); + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + /* Walk the singles. */ + for ( RedTransList::Iter stel = st->outSingle; stel.lte(); stel++ ) { + RedTransAp *trans = stel->value; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ++totalTrans, false ); + } + + /* Walk the ranges. */ + for ( RedTransList::Iter rtel = st->outRange; rtel.lte(); rtel++ ) { + RedTransAp *trans = rtel->value; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ++totalTrans, false ); + } + + /* The state's default index goes next. */ + if ( st->defTrans != 0 ) { + RedTransAp *trans = st->defTrans; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ++totalTrans, false ); + } + } + + for ( RedStateList::Iter st = redFsm->stateList; st.lte(); st++ ) { + if ( st->eofTrans != 0 ) { + RedTransAp *trans = st->eofTrans; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ++totalTrans, false ); + } + } + + /* Output one last number so we don't have to figure out when the last + * entry is and avoid writing a comma. */ + ARRAY_ITEM( INT(0), ++totalTrans, true ); + END_ARRAY_LINE(); + return out; +} + +std::ostream &RubyTabCodeGen::TRANS_TARGS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + START_ARRAY_LINE(); + int totalStates = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Save the position. Needed for eofTargs. */ + RedTransAp *trans = transPtrs[t]; + trans->pos = t; + + /* Write out the target state. */ + ARRAY_ITEM( INT(trans->targ->id), ++totalStates, ( t >= redFsm->transSet.length()-1 ) ); + } + END_ARRAY_LINE(); + delete[] transPtrs; + return out; +} + + +std::ostream &RubyTabCodeGen::TRANS_ACTIONS_WI() +{ + /* Transitions must be written ordered by their id. */ + RedTransAp **transPtrs = new RedTransAp*[redFsm->transSet.length()]; + for ( TransApSet::Iter trans = redFsm->transSet; trans.lte(); trans++ ) + transPtrs[trans->id] = trans; + + /* Keep a count of the num of items in the array written. */ + START_ARRAY_LINE(); + int totalAct = 0; + for ( int t = 0; t < redFsm->transSet.length(); t++ ) { + /* Write the function for the transition. */ + RedTransAp *trans = transPtrs[t]; + ARRAY_ITEM( INT(TRANS_ACTION( trans )), ++totalAct, + ( t >= redFsm->transSet.length()-1 ) ); + } + END_ARRAY_LINE(); + delete[] transPtrs; + return out; +} + + +void RubyTabCodeGen::writeData() +{ + /* If there are any transtion functions then output the array. If there + * are none, don't bother emitting an empty array that won't be used. */ + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActArrItem), A() ); + ACTIONS_ARRAY(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyConditions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondOffset), CO() ); + COND_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondLen), CL() ); + COND_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), CK() ); + COND_KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxCondSpaceId), C() ); + COND_SPACES(); + CLOSE_ARRAY() << + "\n"; + } + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxKeyOffset), KO() ); + KEY_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( WIDE_ALPH_TYPE(), K() ); + KEYS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxSingleLen), SL() ); + SINGLE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxRangeLen), RL() ); + RANGE_LENS(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset), IO() ); + INDEX_OFFSETS(); + CLOSE_ARRAY() << + "\n"; + + if ( useIndicies ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndex), I() ); + INDICIES(); + CLOSE_ARRAY() << + "\n"; + + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS_WI(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS_WI(); + CLOSE_ARRAY() << + "\n"; + } + } + else { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxState), TT() ); + TRANS_TARGS(); + CLOSE_ARRAY() << + "\n"; + + if ( redFsm->anyActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TA() ); + TRANS_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + } + + if ( redFsm->anyToStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), TSA() ); + TO_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyFromStateActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), FSA() ); + FROM_STATE_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofActions() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxActionLoc), EA() ); + EOF_ACTIONS(); + CLOSE_ARRAY() << + "\n"; + } + + if ( redFsm->anyEofTrans() ) { + OPEN_ARRAY( ARRAY_TYPE(redFsm->maxIndexOffset+1), ET() ); + EOF_TRANS(); + CLOSE_ARRAY() << + "\n"; + } + + STATE_IDS(); +} + +/* + Local Variables: + mode: c++ + indent-tabs-mode: 1 + c-file-style: "bsd" + End: + */ diff --git a/contrib/tools/ragel6/rubytable.h b/contrib/tools/ragel6/rubytable.h new file mode 100644 index 0000000000..76b847fdff --- /dev/null +++ b/contrib/tools/ragel6/rubytable.h @@ -0,0 +1,124 @@ +/* + * Copyright 2007 Victor Hugo Borja <vic@rubyforge.org> + * 2006-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _RUBY_TABCODEGEN_H +#define _RUBY_TABCODEGEN_H + +#include <iostream> +#include <string> +#include <stdio.h> +#include "common.h" +#include "gendata.h" +#include "rubycodegen.h" + + +using std::string; +using std::ostream; + +/* + * RubyCodeGen + */ +class RubyTabCodeGen : public RubyCodeGen +{ +public: + RubyTabCodeGen( ostream &out ) : + RubyCodeGen(out) {} + virtual ~RubyTabCodeGen() {} + +public: + void BREAK( ostream &ret, int targState ); + void GOTO( ostream &ret, int gotoDest, bool inFinish ); + void GOTO_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + void CALL( ostream &ret, int callDest, int targState, bool inFinish ); + void CALL_EXPR( ostream &ret, GenInlineItem *ilItem, int targState, bool inFinish ); + void RET( ostream &ret, bool inFinish ); + + void COND_TRANSLATE(); + void LOCATE_TRANS(); + + virtual void writeExec(); + virtual void writeData(); + + protected: + virtual std::ostream &TO_STATE_ACTION_SWITCH(); + virtual std::ostream &FROM_STATE_ACTION_SWITCH(); + virtual std::ostream &EOF_ACTION_SWITCH(); + virtual std::ostream &ACTION_SWITCH(); + + std::ostream &COND_KEYS(); + std::ostream &COND_SPACES(); + std::ostream &KEYS(); + std::ostream &INDICIES(); + std::ostream &COND_OFFSETS(); + std::ostream &KEY_OFFSETS(); + std::ostream &INDEX_OFFSETS(); + std::ostream &COND_LENS(); + std::ostream &SINGLE_LENS(); + std::ostream &RANGE_LENS(); + std::ostream &TO_STATE_ACTIONS(); + std::ostream &FROM_STATE_ACTIONS(); + std::ostream &EOF_ACTIONS(); + std::ostream &EOF_TRANS(); + std::ostream &TRANS_TARGS(); + std::ostream &TRANS_ACTIONS(); + std::ostream &TRANS_TARGS_WI(); + std::ostream &TRANS_ACTIONS_WI(); + + + void NEXT( ostream &ret, int nextDest, bool inFinish ); + void NEXT_EXPR( ostream &ret, GenInlineItem *ilItem, bool inFinish ); + + virtual int TO_STATE_ACTION( RedStateAp *state ); + virtual int FROM_STATE_ACTION( RedStateAp *state ); + virtual int EOF_ACTION( RedStateAp *state ); + +private: + string array_type; + string array_name; + +public: + + void EXEC( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void EXECTE( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void LM_SWITCH( ostream &ret, GenInlineItem *item, int targState, int inFinish ); + void SET_ACT( ostream &ret, GenInlineItem *item ); + void INIT_TOKSTART( ostream &ret, GenInlineItem *item ); + void INIT_ACT( ostream &ret, GenInlineItem *item ); + void SET_TOKSTART( ostream &ret, GenInlineItem *item ); + void SET_TOKEND( ostream &ret, GenInlineItem *item ); + void GET_TOKEND( ostream &ret, GenInlineItem *item ); + void SUB_ACTION( ostream &ret, GenInlineItem *item, + int targState, bool inFinish ); + + +}; + + +#endif + +/* + * Local Variables: + * mode: c++ + * indent-tabs-mode: 1 + * c-file-style: "bsd" + * End: + */ diff --git a/contrib/tools/ragel6/version.h b/contrib/tools/ragel6/version.h new file mode 100644 index 0000000000..3b1233c966 --- /dev/null +++ b/contrib/tools/ragel6/version.h @@ -0,0 +1,2 @@ +#define VERSION "6.10" +#define PUBDATE "March 2017" diff --git a/contrib/tools/ragel6/xmlcodegen.cpp b/contrib/tools/ragel6/xmlcodegen.cpp new file mode 100644 index 0000000000..e4b14bc82b --- /dev/null +++ b/contrib/tools/ragel6/xmlcodegen.cpp @@ -0,0 +1,1429 @@ +/* + * Copyright 2005-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + + +#include "ragel.h" +#include "xmlcodegen.h" +#include "parsedata.h" +#include "fsmgraph.h" +#include "gendata.h" +#include "inputdata.h" +#include <string.h> +#include "rlparse.h" +#include "version.h" + +using namespace std; + +GenBase::GenBase( char *fsmName, ParseData *pd, FsmAp *fsm ) +: + fsmName(fsmName), + pd(pd), + fsm(fsm), + nextActionTableId(0) +{ +} + +void GenBase::appendTrans( TransListVect &outList, Key lowKey, + Key highKey, TransAp *trans ) +{ + if ( trans->toState != 0 || trans->actionTable.length() > 0 ) + outList.append( TransEl( lowKey, highKey, trans ) ); +} + +void GenBase::reduceActionTables() +{ + /* Reduce the actions tables to a set. */ + for ( StateList::Iter st = fsm->stateList; st.lte(); st++ ) { + RedActionTable *actionTable = 0; + + /* Reduce To State Actions. */ + if ( st->toStateActionTable.length() > 0 ) { + if ( actionTableMap.insert( st->toStateActionTable, &actionTable ) ) + actionTable->id = nextActionTableId++; + } + + /* Reduce From State Actions. */ + if ( st->fromStateActionTable.length() > 0 ) { + if ( actionTableMap.insert( st->fromStateActionTable, &actionTable ) ) + actionTable->id = nextActionTableId++; + } + + /* Reduce EOF actions. */ + if ( st->eofActionTable.length() > 0 ) { + if ( actionTableMap.insert( st->eofActionTable, &actionTable ) ) + actionTable->id = nextActionTableId++; + } + + /* Loop the transitions and reduce their actions. */ + for ( TransList::Iter trans = st->outList; trans.lte(); trans++ ) { + if ( trans->actionTable.length() > 0 ) { + if ( actionTableMap.insert( trans->actionTable, &actionTable ) ) + actionTable->id = nextActionTableId++; + } + } + } +} + +XMLCodeGen::XMLCodeGen( char *fsmName, ParseData *pd, FsmAp *fsm, std::ostream &out ) +: + GenBase(fsmName, pd, fsm), + out(out) +{ +} + + +void XMLCodeGen::writeActionList() +{ + /* Determine which actions to write. */ + int nextActionId = 0; + for ( ActionList::Iter act = pd->actionList; act.lte(); act++ ) { + if ( act->numRefs() > 0 || act->numCondRefs > 0 ) + act->actionId = nextActionId++; + } + + /* Write the list. */ + out << " <action_list length=\"" << nextActionId << "\">\n"; + for ( ActionList::Iter act = pd->actionList; act.lte(); act++ ) { + if ( act->actionId >= 0 ) + writeAction( act ); + } + out << " </action_list>\n"; +} + +void XMLCodeGen::writeActionTableList() +{ + /* Must first order the action tables based on their id. */ + int numTables = nextActionTableId; + RedActionTable **tables = new RedActionTable*[numTables]; + for ( ActionTableMap::Iter at = actionTableMap; at.lte(); at++ ) + tables[at->id] = at; + + out << " <action_table_list length=\"" << numTables << "\">\n"; + for ( int t = 0; t < numTables; t++ ) { + out << " <action_table id=\"" << t << "\" length=\"" << + tables[t]->key.length() << "\">"; + for ( ActionTable::Iter atel = tables[t]->key; atel.lte(); atel++ ) { + out << atel->value->actionId; + if ( ! atel.last() ) + out << " "; + } + out << "</action_table>\n"; + } + out << " </action_table_list>\n"; + + delete[] tables; +} + +void XMLCodeGen::writeKey( Key key ) +{ + if ( keyOps->isSigned ) + out << key.getVal(); + else + out << (unsigned long) key.getVal(); +} + +void XMLCodeGen::writeTrans( Key lowKey, Key highKey, TransAp *trans ) +{ + /* First reduce the action. */ + RedActionTable *actionTable = 0; + if ( trans->actionTable.length() > 0 ) + actionTable = actionTableMap.find( trans->actionTable ); + + /* Write the transition. */ + out << " <t>"; + writeKey( lowKey ); + out << " "; + writeKey( highKey ); + + if ( trans->toState != 0 ) + out << " " << trans->toState->alg.stateNum; + else + out << " x"; + + if ( actionTable != 0 ) + out << " " << actionTable->id; + else + out << " x"; + out << "</t>\n"; +} + +void XMLCodeGen::writeTransList( StateAp *state ) +{ + TransListVect outList; + + /* If there is only are no ranges the task is simple. */ + if ( state->outList.length() > 0 ) { + /* Loop each source range. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + /* Reduce the transition. If it reduced to anything then add it. */ + appendTrans( outList, trans->lowKey, trans->highKey, trans ); + } + } + + out << " <trans_list length=\"" << outList.length() << "\">\n"; + for ( TransListVect::Iter tvi = outList; tvi.lte(); tvi++ ) + writeTrans( tvi->lowKey, tvi->highKey, tvi->value ); + out << " </trans_list>\n"; +} + +void XMLCodeGen::writeEofTrans( StateAp *state ) +{ + RedActionTable *eofActions = 0; + if ( state->eofActionTable.length() > 0 ) + eofActions = actionTableMap.find( state->eofActionTable ); + + /* The <eof_t> is used when there is an eof target, otherwise the eof + * action goes into state actions. */ + if ( state->eofTarget != 0 ) { + out << " <eof_t>" << state->eofTarget->alg.stateNum; + + if ( eofActions != 0 ) + out << " " << eofActions->id; + else + out << " x"; + + out << "</eof_t>" << endl; + } +} + +void XMLCodeGen::writeText( InlineItem *item ) +{ + if ( item->prev == 0 || item->prev->type != InlineItem::Text ) + out << "<text>"; + xmlEscapeHost( out, item->data, strlen(item->data) ); + if ( item->next == 0 || item->next->type != InlineItem::Text ) + out << "</text>"; +} + +void XMLCodeGen::writeGoto( InlineItem *item ) +{ + if ( pd->generatingSectionSubset ) + out << "<goto>-1</goto>"; + else { + EntryMapEl *targ = fsm->entryPoints.find( item->nameTarg->id ); + out << "<goto>" << targ->value->alg.stateNum << "</goto>"; + } +} + +void XMLCodeGen::writeCall( InlineItem *item ) +{ + if ( pd->generatingSectionSubset ) + out << "<call>-1</call>"; + else { + EntryMapEl *targ = fsm->entryPoints.find( item->nameTarg->id ); + out << "<call>" << targ->value->alg.stateNum << "</call>"; + } +} + +void XMLCodeGen::writeNext( InlineItem *item ) +{ + if ( pd->generatingSectionSubset ) + out << "<next>-1</next>"; + else { + EntryMapEl *targ = fsm->entryPoints.find( item->nameTarg->id ); + out << "<next>" << targ->value->alg.stateNum << "</next>"; + } +} + +void XMLCodeGen::writeGotoExpr( InlineItem *item ) +{ + out << "<goto_expr>"; + writeInlineList( item->children ); + out << "</goto_expr>"; +} + +void XMLCodeGen::writeCallExpr( InlineItem *item ) +{ + out << "<call_expr>"; + writeInlineList( item->children ); + out << "</call_expr>"; +} + +void XMLCodeGen::writeNextExpr( InlineItem *item ) +{ + out << "<next_expr>"; + writeInlineList( item->children ); + out << "</next_expr>"; +} + +void XMLCodeGen::writeEntry( InlineItem *item ) +{ + if ( pd->generatingSectionSubset ) + out << "<entry>-1</entry>"; + else { + EntryMapEl *targ = fsm->entryPoints.find( item->nameTarg->id ); + out << "<entry>" << targ->value->alg.stateNum << "</entry>"; + } +} + +void XMLCodeGen::writeActionExec( InlineItem *item ) +{ + out << "<exec>"; + writeInlineList( item->children ); + out << "</exec>"; +} + +void XMLCodeGen::writeLmOnLast( InlineItem *item ) +{ + out << "<set_tokend>1</set_tokend>"; + + if ( item->longestMatchPart->action != 0 ) { + out << "<sub_action>"; + writeInlineList( item->longestMatchPart->action->inlineList ); + out << "</sub_action>"; + } +} + +void XMLCodeGen::writeLmOnNext( InlineItem *item ) +{ + out << "<set_tokend>0</set_tokend>"; + out << "<hold></hold>"; + + if ( item->longestMatchPart->action != 0 ) { + out << "<sub_action>"; + writeInlineList( item->longestMatchPart->action->inlineList ); + out << "</sub_action>"; + } +} + +void XMLCodeGen::writeLmOnLagBehind( InlineItem *item ) +{ + out << "<exec><get_tokend></get_tokend></exec>"; + + if ( item->longestMatchPart->action != 0 ) { + out << "<sub_action>"; + writeInlineList( item->longestMatchPart->action->inlineList ); + out << "</sub_action>"; + } +} + +void XMLCodeGen::writeLmSwitch( InlineItem *item ) +{ + LongestMatch *longestMatch = item->longestMatch; + out << "<lm_switch>\n"; + + /* We can't put the <exec> here because we may need to handle the error + * case and in that case p should not be changed. Instead use a default + * label in the switch to adjust p when user actions are not set. An id of + * -1 indicates the default. */ + + if ( longestMatch->lmSwitchHandlesError ) { + /* If the switch handles error then we should have also forced the + * error state. */ + assert( fsm->errState != 0 ); + + out << " <sub_action id=\"0\">"; + out << "<goto>" << fsm->errState->alg.stateNum << "</goto>"; + out << "</sub_action>\n"; + } + + bool needDefault = false; + for ( LmPartList::Iter lmi = *longestMatch->longestMatchList; lmi.lte(); lmi++ ) { + if ( lmi->inLmSelect ) { + if ( lmi->action == 0 ) + needDefault = true; + else { + /* Open the action. Write it with the context that sets up _p + * when doing control flow changes from inside the machine. */ + out << " <sub_action id=\"" << lmi->longestMatchId << "\">"; + out << "<exec><get_tokend></get_tokend></exec>"; + writeInlineList( lmi->action->inlineList ); + out << "</sub_action>\n"; + } + } + } + + if ( needDefault ) { + out << " <sub_action id=\"-1\"><exec><get_tokend>" + "</get_tokend></exec></sub_action>\n"; + } + + out << " </lm_switch>"; +} + +void XMLCodeGen::writeInlineList( InlineList *inlineList ) +{ + for ( InlineList::Iter item = *inlineList; item.lte(); item++ ) { + switch ( item->type ) { + case InlineItem::Text: + writeText( item ); + break; + case InlineItem::Goto: + writeGoto( item ); + break; + case InlineItem::GotoExpr: + writeGotoExpr( item ); + break; + case InlineItem::Call: + writeCall( item ); + break; + case InlineItem::CallExpr: + writeCallExpr( item ); + break; + case InlineItem::Next: + writeNext( item ); + break; + case InlineItem::NextExpr: + writeNextExpr( item ); + break; + case InlineItem::Break: + out << "<break></break>"; + break; + case InlineItem::Ret: + out << "<ret></ret>"; + break; + case InlineItem::PChar: + out << "<pchar></pchar>"; + break; + case InlineItem::Char: + out << "<char></char>"; + break; + case InlineItem::Curs: + out << "<curs></curs>"; + break; + case InlineItem::Targs: + out << "<targs></targs>"; + break; + case InlineItem::Entry: + writeEntry( item ); + break; + + case InlineItem::Hold: + out << "<hold></hold>"; + break; + case InlineItem::Exec: + writeActionExec( item ); + break; + + case InlineItem::LmSetActId: + out << "<set_act>" << + item->longestMatchPart->longestMatchId << + "</set_act>"; + break; + case InlineItem::LmSetTokEnd: + out << "<set_tokend>1</set_tokend>"; + break; + + case InlineItem::LmOnLast: + writeLmOnLast( item ); + break; + case InlineItem::LmOnNext: + writeLmOnNext( item ); + break; + case InlineItem::LmOnLagBehind: + writeLmOnLagBehind( item ); + break; + case InlineItem::LmSwitch: + writeLmSwitch( item ); + break; + + case InlineItem::LmInitAct: + out << "<init_act></init_act>"; + break; + case InlineItem::LmInitTokStart: + out << "<init_tokstart></init_tokstart>"; + break; + case InlineItem::LmSetTokStart: + out << "<set_tokstart></set_tokstart>"; + break; + } + } +} + +BackendGen::BackendGen( char *fsmName, ParseData *pd, FsmAp *fsm, CodeGenData *cgd ) +: + GenBase(fsmName, pd, fsm), + cgd(cgd) +{ +} + + +void BackendGen::makeText( GenInlineList *outList, InlineItem *item ) +{ + GenInlineItem *inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::Text ); + inlineItem->data = item->data; + + outList->append( inlineItem ); +} + +void BackendGen::makeTargetItem( GenInlineList *outList, NameInst *nameTarg, + GenInlineItem::Type type ) +{ + long targetState; + if ( pd->generatingSectionSubset ) + targetState = -1; + else { + EntryMapEl *targ = fsm->entryPoints.find( nameTarg->id ); + targetState = targ->value->alg.stateNum; + } + + /* Make the item. */ + GenInlineItem *inlineItem = new GenInlineItem( InputLoc(), type ); + inlineItem->targId = targetState; + outList->append( inlineItem ); +} + +/* Make a sublist item with a given type. */ +void BackendGen::makeSubList( GenInlineList *outList, + InlineList *inlineList, GenInlineItem::Type type ) +{ + /* Fill the sub list. */ + GenInlineList *subList = new GenInlineList; + makeGenInlineList( subList, inlineList ); + + /* Make the item. */ + GenInlineItem *inlineItem = new GenInlineItem( InputLoc(), type ); + inlineItem->children = subList; + outList->append( inlineItem ); +} + +void BackendGen::makeLmOnLast( GenInlineList *outList, InlineItem *item ) +{ + makeSetTokend( outList, 1 ); + + if ( item->longestMatchPart->action != 0 ) { + makeSubList( outList, + item->longestMatchPart->action->inlineList, + GenInlineItem::SubAction ); + } +} + +void BackendGen::makeLmOnNext( GenInlineList *outList, InlineItem *item ) +{ + makeSetTokend( outList, 0 ); + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Hold ) ); + + if ( item->longestMatchPart->action != 0 ) { + makeSubList( outList, + item->longestMatchPart->action->inlineList, + GenInlineItem::SubAction ); + } +} + +void BackendGen::makeExecGetTokend( GenInlineList *outList ) +{ + /* Make the Exec item. */ + GenInlineItem *execItem = new GenInlineItem( InputLoc(), GenInlineItem::Exec ); + execItem->children = new GenInlineList; + + /* Make the GetTokEnd */ + GenInlineItem *getTokend = new GenInlineItem( InputLoc(), GenInlineItem::LmGetTokEnd ); + execItem->children->append( getTokend ); + + outList->append( execItem ); +} + +void BackendGen::makeLmOnLagBehind( GenInlineList *outList, InlineItem *item ) +{ + /* Jump to the tokend. */ + makeExecGetTokend( outList ); + + if ( item->longestMatchPart->action != 0 ) { + makeSubList( outList, + item->longestMatchPart->action->inlineList, + GenInlineItem::SubAction ); + } +} + +void BackendGen::makeLmSwitch( GenInlineList *outList, InlineItem *item ) +{ + GenInlineItem *lmSwitch = new GenInlineItem( InputLoc(), GenInlineItem::LmSwitch ); + GenInlineList *lmList = lmSwitch->children = new GenInlineList; + LongestMatch *longestMatch = item->longestMatch; + + /* We can't put the <exec> here because we may need to handle the error + * case and in that case p should not be changed. Instead use a default + * label in the switch to adjust p when user actions are not set. An id of + * -1 indicates the default. */ + + if ( longestMatch->lmSwitchHandlesError ) { + /* If the switch handles error then we should have also forced the + * error state. */ + assert( fsm->errState != 0 ); + + GenInlineItem *errCase = new GenInlineItem( InputLoc(), GenInlineItem::SubAction ); + errCase->lmId = 0; + errCase->children = new GenInlineList; + + /* Make the item. */ + GenInlineItem *gotoItem = new GenInlineItem( InputLoc(), GenInlineItem::Goto ); + gotoItem->targId = fsm->errState->alg.stateNum; + errCase->children->append( gotoItem ); + + lmList->append( errCase ); + } + + bool needDefault = false; + for ( LmPartList::Iter lmi = *longestMatch->longestMatchList; lmi.lte(); lmi++ ) { + if ( lmi->inLmSelect ) { + if ( lmi->action == 0 ) + needDefault = true; + else { + /* Open the action. Write it with the context that sets up _p + * when doing control flow changes from inside the machine. */ + GenInlineItem *lmCase = new GenInlineItem( InputLoc(), + GenInlineItem::SubAction ); + lmCase->lmId = lmi->longestMatchId; + lmCase->children = new GenInlineList; + + makeExecGetTokend( lmCase->children ); + makeGenInlineList( lmCase->children, lmi->action->inlineList ); + + lmList->append( lmCase ); + } + } + } + + if ( needDefault ) { + GenInlineItem *defCase = new GenInlineItem( InputLoc(), + GenInlineItem::SubAction ); + defCase->lmId = -1; + defCase->children = new GenInlineList; + + makeExecGetTokend( defCase->children ); + + lmList->append( defCase ); + } + + outList->append( lmSwitch ); +} + +void BackendGen::makeSetTokend( GenInlineList *outList, long offset ) +{ + GenInlineItem *inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmSetTokEnd ); + inlineItem->offset = offset; + outList->append( inlineItem ); +} + +void BackendGen::makeSetAct( GenInlineList *outList, long lmId ) +{ + GenInlineItem *inlineItem = new GenInlineItem( InputLoc(), GenInlineItem::LmSetActId ); + inlineItem->lmId = lmId; + outList->append( inlineItem ); +} + +void BackendGen::makeGenInlineList( GenInlineList *outList, InlineList *inList ) +{ + for ( InlineList::Iter item = *inList; item.lte(); item++ ) { + switch ( item->type ) { + case InlineItem::Text: + makeText( outList, item ); + break; + case InlineItem::Goto: + makeTargetItem( outList, item->nameTarg, GenInlineItem::Goto ); + break; + case InlineItem::GotoExpr: + makeSubList( outList, item->children, GenInlineItem::GotoExpr ); + break; + case InlineItem::Call: + makeTargetItem( outList, item->nameTarg, GenInlineItem::Call ); + break; + case InlineItem::CallExpr: + makeSubList( outList, item->children, GenInlineItem::CallExpr ); + break; + case InlineItem::Next: + makeTargetItem( outList, item->nameTarg, GenInlineItem::Next ); + break; + case InlineItem::NextExpr: + makeSubList( outList, item->children, GenInlineItem::NextExpr ); + break; + case InlineItem::Break: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Break ) ); + break; + case InlineItem::Ret: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Ret ) ); + break; + case InlineItem::PChar: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::PChar ) ); + break; + case InlineItem::Char: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Char ) ); + break; + case InlineItem::Curs: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Curs ) ); + break; + case InlineItem::Targs: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Targs ) ); + break; + case InlineItem::Entry: + makeTargetItem( outList, item->nameTarg, GenInlineItem::Entry ); + break; + + case InlineItem::Hold: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::Hold ) ); + break; + case InlineItem::Exec: + makeSubList( outList, item->children, GenInlineItem::Exec ); + break; + + case InlineItem::LmSetActId: + makeSetAct( outList, item->longestMatchPart->longestMatchId ); + break; + case InlineItem::LmSetTokEnd: + makeSetTokend( outList, 1 ); + break; + + case InlineItem::LmOnLast: + makeLmOnLast( outList, item ); + break; + case InlineItem::LmOnNext: + makeLmOnNext( outList, item ); + break; + case InlineItem::LmOnLagBehind: + makeLmOnLagBehind( outList, item ); + break; + case InlineItem::LmSwitch: + makeLmSwitch( outList, item ); + break; + + case InlineItem::LmInitAct: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::LmInitAct ) ); + break; + case InlineItem::LmInitTokStart: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::LmInitTokStart ) ); + break; + case InlineItem::LmSetTokStart: + outList->append( new GenInlineItem( InputLoc(), GenInlineItem::LmSetTokStart ) ); + cgd->hasLongestMatch = true; + break; + } + } +} + + +void XMLCodeGen::writeAction( Action *action ) +{ + out << " <action id=\"" << action->actionId << "\""; + if ( action->name != 0 ) + out << " name=\"" << action->name << "\""; + out << " line=\"" << action->loc.line << "\" col=\"" << action->loc.col << "\">"; + writeInlineList( action->inlineList ); + out << "</action>\n"; +} + +void xmlEscapeHost( std::ostream &out, char *data, long len ) +{ + char *end = data + len; + while ( data != end ) { + switch ( *data ) { + case '<': out << "<"; break; + case '>': out << ">"; break; + case '&': out << "&"; break; + default: out << *data; break; + } + data += 1; + } +} + +void XMLCodeGen::writeStateActions( StateAp *state ) +{ + RedActionTable *toStateActions = 0; + if ( state->toStateActionTable.length() > 0 ) + toStateActions = actionTableMap.find( state->toStateActionTable ); + + RedActionTable *fromStateActions = 0; + if ( state->fromStateActionTable.length() > 0 ) + fromStateActions = actionTableMap.find( state->fromStateActionTable ); + + /* EOF actions go out here only if the state has no eof target. If it has + * an eof target then an eof transition will be used instead. */ + RedActionTable *eofActions = 0; + if ( state->eofTarget == 0 && state->eofActionTable.length() > 0 ) + eofActions = actionTableMap.find( state->eofActionTable ); + + if ( toStateActions != 0 || fromStateActions != 0 || eofActions != 0 ) { + out << " <state_actions>"; + if ( toStateActions != 0 ) + out << toStateActions->id; + else + out << "x"; + + if ( fromStateActions != 0 ) + out << " " << fromStateActions->id; + else + out << " x"; + + if ( eofActions != 0 ) + out << " " << eofActions->id; + else + out << " x"; + + out << "</state_actions>\n"; + } +} + +void XMLCodeGen::writeStateConditions( StateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) { + out << " <cond_list length=\"" << state->stateCondList.length() << "\">\n"; + for ( StateCondList::Iter scdi = state->stateCondList; scdi.lte(); scdi++ ) { + out << " <c>"; + writeKey( scdi->lowKey ); + out << " "; + writeKey( scdi->highKey ); + out << " "; + out << scdi->condSpace->condSpaceId; + out << "</c>\n"; + } + out << " </cond_list>\n"; + } +} + +void XMLCodeGen::writeStateList() +{ + /* Write the list of states. */ + out << " <state_list length=\"" << fsm->stateList.length() << "\">\n"; + for ( StateList::Iter st = fsm->stateList; st.lte(); st++ ) { + out << " <state id=\"" << st->alg.stateNum << "\""; + if ( st->isFinState() ) + out << " final=\"t\""; + out << ">\n"; + + writeStateActions( st ); + writeEofTrans( st ); + writeStateConditions( st ); + writeTransList( st ); + + out << " </state>\n"; + + if ( !st.last() ) + out << "\n"; + } + out << " </state_list>\n"; +} + +bool XMLCodeGen::writeNameInst( NameInst *nameInst ) +{ + bool written = false; + if ( nameInst->parent != 0 ) + written = writeNameInst( nameInst->parent ); + + if ( nameInst->name != 0 ) { + if ( written ) + out << '_'; + out << nameInst->name; + written = true; + } + + return written; +} + +void XMLCodeGen::writeEntryPoints() +{ + /* List of entry points other than start state. */ + if ( fsm->entryPoints.length() > 0 || pd->lmRequiresErrorState ) { + out << " <entry_points"; + if ( pd->lmRequiresErrorState ) + out << " error=\"t\""; + out << ">\n"; + for ( EntryMap::Iter en = fsm->entryPoints; en.lte(); en++ ) { + /* Get the name instantiation from nameIndex. */ + NameInst *nameInst = pd->nameIndex[en->key]; + StateAp *state = en->value; + out << " <entry name=\""; + writeNameInst( nameInst ); + out << "\">" << state->alg.stateNum << "</entry>\n"; + } + out << " </entry_points>\n"; + } +} + +void XMLCodeGen::writeMachine() +{ + /* Open the machine. */ + out << " <machine>\n"; + + /* Action tables. */ + reduceActionTables(); + + writeActionList(); + writeActionTableList(); + writeConditions(); + + /* Start state. */ + out << " <start_state>" << fsm->startState->alg.stateNum << + "</start_state>\n"; + + /* Error state. */ + if ( fsm->errState != 0 ) { + out << " <error_state>" << fsm->errState->alg.stateNum << + "</error_state>\n"; + } + + writeEntryPoints(); + writeStateList(); + + out << " </machine>\n"; +} + + +void XMLCodeGen::writeConditions() +{ + if ( condData->condSpaceMap.length() > 0 ) { + long nextCondSpaceId = 0; + for ( CondSpaceMap::Iter cs = condData->condSpaceMap; cs.lte(); cs++ ) + cs->condSpaceId = nextCondSpaceId++; + + out << " <cond_space_list length=\"" << condData->condSpaceMap.length() << "\">\n"; + for ( CondSpaceMap::Iter cs = condData->condSpaceMap; cs.lte(); cs++ ) { + out << " <cond_space id=\"" << cs->condSpaceId << + "\" length=\"" << cs->condSet.length() << "\">"; + writeKey( cs->baseKey ); + for ( CondSet::Iter csi = cs->condSet; csi.lte(); csi++ ) + out << " " << (*csi)->actionId; + out << "</cond_space>\n"; + } + out << " </cond_space_list>\n"; + } +} + +void XMLCodeGen::writeExports() +{ + if ( pd->exportList.length() > 0 ) { + out << " <exports>\n"; + for ( ExportList::Iter exp = pd->exportList; exp.lte(); exp++ ) { + out << " <ex name=\"" << exp->name << "\">"; + writeKey( exp->key ); + out << "</ex>\n"; + } + out << " </exports>\n"; + } +} + +void XMLCodeGen::writeXML() +{ + /* Open the definition. */ + out << "<ragel_def name=\"" << fsmName << "\">\n"; + + /* Alphabet type. */ + out << " <alphtype>" << keyOps->alphType->internalName << "</alphtype>\n"; + + /* Getkey expression. */ + if ( pd->getKeyExpr != 0 ) { + out << " <getkey>"; + writeInlineList( pd->getKeyExpr ); + out << "</getkey>\n"; + } + + /* Access expression. */ + if ( pd->accessExpr != 0 ) { + out << " <access>"; + writeInlineList( pd->accessExpr ); + out << "</access>\n"; + } + + /* PrePush expression. */ + if ( pd->prePushExpr != 0 ) { + out << " <prepush>"; + writeInlineList( pd->prePushExpr ); + out << "</prepush>\n"; + } + + /* PostPop expression. */ + if ( pd->postPopExpr != 0 ) { + out << " <postpop>"; + writeInlineList( pd->postPopExpr ); + out << "</postpop>\n"; + } + + /* + * Variable expressions. + */ + + if ( pd->pExpr != 0 ) { + out << " <p_expr>"; + writeInlineList( pd->pExpr ); + out << "</p_expr>\n"; + } + + if ( pd->peExpr != 0 ) { + out << " <pe_expr>"; + writeInlineList( pd->peExpr ); + out << "</pe_expr>\n"; + } + + if ( pd->eofExpr != 0 ) { + out << " <eof_expr>"; + writeInlineList( pd->eofExpr ); + out << "</eof_expr>\n"; + } + + if ( pd->csExpr != 0 ) { + out << " <cs_expr>"; + writeInlineList( pd->csExpr ); + out << "</cs_expr>\n"; + } + + if ( pd->topExpr != 0 ) { + out << " <top_expr>"; + writeInlineList( pd->topExpr ); + out << "</top_expr>\n"; + } + + if ( pd->stackExpr != 0 ) { + out << " <stack_expr>"; + writeInlineList( pd->stackExpr ); + out << "</stack_expr>\n"; + } + + if ( pd->actExpr != 0 ) { + out << " <act_expr>"; + writeInlineList( pd->actExpr ); + out << "</act_expr>\n"; + } + + if ( pd->tokstartExpr != 0 ) { + out << " <tokstart_expr>"; + writeInlineList( pd->tokstartExpr ); + out << "</tokstart_expr>\n"; + } + + if ( pd->tokendExpr != 0 ) { + out << " <tokend_expr>"; + writeInlineList( pd->tokendExpr ); + out << "</tokend_expr>\n"; + } + + if ( pd->dataExpr != 0 ) { + out << " <data_expr>"; + writeInlineList( pd->dataExpr ); + out << "</data_expr>\n"; + } + + writeExports(); + + writeMachine(); + + out << + "</ragel_def>\n"; +} + +void BackendGen::makeExports() +{ + for ( ExportList::Iter exp = pd->exportList; exp.lte(); exp++ ) + cgd->exportList.append( new Export( exp->name, exp->key ) ); +} + +void BackendGen::makeAction( Action *action ) +{ + GenInlineList *genList = new GenInlineList; + makeGenInlineList( genList, action->inlineList ); + + cgd->newAction( curAction++, action->name, action->loc, genList ); +} + + +void BackendGen::makeActionList() +{ + /* Determine which actions to write. */ + int nextActionId = 0; + for ( ActionList::Iter act = pd->actionList; act.lte(); act++ ) { + if ( act->numRefs() > 0 || act->numCondRefs > 0 ) + act->actionId = nextActionId++; + } + + /* Write the list. */ + cgd->initActionList( nextActionId ); + curAction = 0; + + for ( ActionList::Iter act = pd->actionList; act.lte(); act++ ) { + if ( act->actionId >= 0 ) + makeAction( act ); + } +} + +void BackendGen::makeActionTableList() +{ + /* Must first order the action tables based on their id. */ + int numTables = nextActionTableId; + RedActionTable **tables = new RedActionTable*[numTables]; + for ( ActionTableMap::Iter at = actionTableMap; at.lte(); at++ ) + tables[at->id] = at; + + cgd->initActionTableList( numTables ); + curActionTable = 0; + + for ( int t = 0; t < numTables; t++ ) { + long length = tables[t]->key.length(); + + /* Collect the action table. */ + RedAction *redAct = cgd->allActionTables + curActionTable; + redAct->actListId = curActionTable; + redAct->key.setAsNew( length ); + + for ( ActionTable::Iter atel = tables[t]->key; atel.lte(); atel++ ) { + redAct->key[atel.pos()].key = 0; + redAct->key[atel.pos()].value = cgd->allActions + + atel->value->actionId; + } + + /* Insert into the action table map. */ + cgd->redFsm->actionMap.insert( redAct ); + + curActionTable += 1; + } + + delete[] tables; +} + +void BackendGen::makeConditions() +{ + if ( condData->condSpaceMap.length() > 0 ) { + long nextCondSpaceId = 0; + for ( CondSpaceMap::Iter cs = condData->condSpaceMap; cs.lte(); cs++ ) + cs->condSpaceId = nextCondSpaceId++; + + long listLength = condData->condSpaceMap.length(); + cgd->initCondSpaceList( listLength ); + curCondSpace = 0; + + for ( CondSpaceMap::Iter cs = condData->condSpaceMap; cs.lte(); cs++ ) { + long id = cs->condSpaceId; + cgd->newCondSpace( curCondSpace, id, cs->baseKey ); + for ( CondSet::Iter csi = cs->condSet; csi.lte(); csi++ ) + cgd->condSpaceItem( curCondSpace, (*csi)->actionId ); + curCondSpace += 1; + } + } +} + +bool BackendGen::makeNameInst( std::string &res, NameInst *nameInst ) +{ + bool written = false; + if ( nameInst->parent != 0 ) + written = makeNameInst( res, nameInst->parent ); + + if ( nameInst->name != 0 ) { + if ( written ) + res += '_'; + res += nameInst->name; + written = true; + } + + return written; +} + +void BackendGen::makeEntryPoints() +{ + /* List of entry points other than start state. */ + if ( fsm->entryPoints.length() > 0 || pd->lmRequiresErrorState ) { + if ( pd->lmRequiresErrorState ) + cgd->setForcedErrorState(); + + for ( EntryMap::Iter en = fsm->entryPoints; en.lte(); en++ ) { + /* Get the name instantiation from nameIndex. */ + NameInst *nameInst = pd->nameIndex[en->key]; + std::string name; + makeNameInst( name, nameInst ); + StateAp *state = en->value; + cgd->addEntryPoint( strdup(name.c_str()), state->alg.stateNum ); + } + } +} + +void BackendGen::makeStateActions( StateAp *state ) +{ + RedActionTable *toStateActions = 0; + if ( state->toStateActionTable.length() > 0 ) + toStateActions = actionTableMap.find( state->toStateActionTable ); + + RedActionTable *fromStateActions = 0; + if ( state->fromStateActionTable.length() > 0 ) + fromStateActions = actionTableMap.find( state->fromStateActionTable ); + + /* EOF actions go out here only if the state has no eof target. If it has + * an eof target then an eof transition will be used instead. */ + RedActionTable *eofActions = 0; + if ( state->eofTarget == 0 && state->eofActionTable.length() > 0 ) + eofActions = actionTableMap.find( state->eofActionTable ); + + if ( toStateActions != 0 || fromStateActions != 0 || eofActions != 0 ) { + long to = -1; + if ( toStateActions != 0 ) + to = toStateActions->id; + + long from = -1; + if ( fromStateActions != 0 ) + from = fromStateActions->id; + + long eof = -1; + if ( eofActions != 0 ) + eof = eofActions->id; + + cgd->setStateActions( curState, to, from, eof ); + } +} + +void BackendGen::makeEofTrans( StateAp *state ) +{ + RedActionTable *eofActions = 0; + if ( state->eofActionTable.length() > 0 ) + eofActions = actionTableMap.find( state->eofActionTable ); + + /* The EOF trans is used when there is an eof target, otherwise the eof + * action goes into state actions. */ + if ( state->eofTarget != 0 ) { + long targ = state->eofTarget->alg.stateNum; + long action = -1; + if ( eofActions != 0 ) + action = eofActions->id; + + cgd->setEofTrans( curState, targ, action ); + } +} + +void BackendGen::makeStateConditions( StateAp *state ) +{ + if ( state->stateCondList.length() > 0 ) { + long length = state->stateCondList.length(); + cgd->initStateCondList( curState, length ); + curStateCond = 0; + + for ( StateCondList::Iter scdi = state->stateCondList; scdi.lte(); scdi++ ) { + cgd->addStateCond( curState, scdi->lowKey, scdi->highKey, + scdi->condSpace->condSpaceId ); + } + } +} + +void BackendGen::makeTrans( Key lowKey, Key highKey, TransAp *trans ) +{ + /* First reduce the action. */ + RedActionTable *actionTable = 0; + if ( trans->actionTable.length() > 0 ) + actionTable = actionTableMap.find( trans->actionTable ); + + long targ = -1; + if ( trans->toState != 0 ) + targ = trans->toState->alg.stateNum; + + long action = -1; + if ( actionTable != 0 ) + action = actionTable->id; + + cgd->newTrans( curState, curTrans++, lowKey, highKey, targ, action ); +} + +void BackendGen::makeTransList( StateAp *state ) +{ + TransListVect outList; + + /* If there is only are no ranges the task is simple. */ + if ( state->outList.length() > 0 ) { + /* Loop each source range. */ + for ( TransList::Iter trans = state->outList; trans.lte(); trans++ ) { + /* Reduce the transition. If it reduced to anything then add it. */ + appendTrans( outList, trans->lowKey, trans->highKey, trans ); + } + } + + cgd->initTransList( curState, outList.length() ); + curTrans = 0; + + for ( TransListVect::Iter tvi = outList; tvi.lte(); tvi++ ) + makeTrans( tvi->lowKey, tvi->highKey, tvi->value ); + + cgd->finishTransList( curState ); +} + + +void BackendGen::makeStateList() +{ + /* Write the list of states. */ + long length = fsm->stateList.length(); + cgd->initStateList( length ); + curState = 0; + for ( StateList::Iter st = fsm->stateList; st.lte(); st++ ) { + makeStateActions( st ); + makeEofTrans( st ); + makeStateConditions( st ); + makeTransList( st ); + + long id = st->alg.stateNum; + cgd->setId( curState, id ); + + if ( st->isFinState() ) + cgd->setFinal( curState ); + + curState += 1; + } +} + + +void BackendGen::makeMachine() +{ + cgd->createMachine(); + + /* Action tables. */ + reduceActionTables(); + + makeActionList(); + makeActionTableList(); + makeConditions(); + + /* Start State. */ + cgd->setStartState( fsm->startState->alg.stateNum ); + + /* Error state. */ + if ( fsm->errState != 0 ) + cgd->setErrorState( fsm->errState->alg.stateNum ); + + makeEntryPoints(); + makeStateList(); + + cgd->closeMachine(); +} + +void BackendGen::close_ragel_def() +{ + /* Do this before distributing transitions out to singles and defaults + * makes life easier. */ + cgd->redFsm->maxKey = cgd->findMaxKey(); + + cgd->redFsm->assignActionLocs(); + + /* Find the first final state (The final state with the lowest id). */ + cgd->redFsm->findFirstFinState(); + + /* Call the user's callback. */ + cgd->finishRagelDef(); +} + + +void BackendGen::makeBackend() +{ + /* Alphabet type. */ + cgd->setAlphType( keyOps->alphType->internalName ); + + /* Getkey expression. */ + if ( pd->getKeyExpr != 0 ) { + cgd->getKeyExpr = new GenInlineList; + makeGenInlineList( cgd->getKeyExpr, pd->getKeyExpr ); + } + + /* Access expression. */ + if ( pd->accessExpr != 0 ) { + cgd->accessExpr = new GenInlineList; + makeGenInlineList( cgd->accessExpr, pd->accessExpr ); + } + + /* PrePush expression. */ + if ( pd->prePushExpr != 0 ) { + cgd->prePushExpr = new GenInlineList; + makeGenInlineList( cgd->prePushExpr, pd->prePushExpr ); + } + + /* PostPop expression. */ + if ( pd->postPopExpr != 0 ) { + cgd->postPopExpr = new GenInlineList; + makeGenInlineList( cgd->postPopExpr, pd->postPopExpr ); + } + + /* + * Variable expressions. + */ + + if ( pd->pExpr != 0 ) { + cgd->pExpr = new GenInlineList; + makeGenInlineList( cgd->pExpr, pd->pExpr ); + } + + if ( pd->peExpr != 0 ) { + cgd->peExpr = new GenInlineList; + makeGenInlineList( cgd->peExpr, pd->peExpr ); + } + + if ( pd->eofExpr != 0 ) { + cgd->eofExpr = new GenInlineList; + makeGenInlineList( cgd->eofExpr, pd->eofExpr ); + } + + if ( pd->csExpr != 0 ) { + cgd->csExpr = new GenInlineList; + makeGenInlineList( cgd->csExpr, pd->csExpr ); + } + + if ( pd->topExpr != 0 ) { + cgd->topExpr = new GenInlineList; + makeGenInlineList( cgd->topExpr, pd->topExpr ); + } + + if ( pd->stackExpr != 0 ) { + cgd->stackExpr = new GenInlineList; + makeGenInlineList( cgd->stackExpr, pd->stackExpr ); + } + + if ( pd->actExpr != 0 ) { + cgd->actExpr = new GenInlineList; + makeGenInlineList( cgd->actExpr, pd->actExpr ); + } + + if ( pd->tokstartExpr != 0 ) { + cgd->tokstartExpr = new GenInlineList; + makeGenInlineList( cgd->tokstartExpr, pd->tokstartExpr ); + } + + if ( pd->tokendExpr != 0 ) { + cgd->tokendExpr = new GenInlineList; + makeGenInlineList( cgd->tokendExpr, pd->tokendExpr ); + } + + if ( pd->dataExpr != 0 ) { + cgd->dataExpr = new GenInlineList; + makeGenInlineList( cgd->dataExpr, pd->dataExpr ); + } + + makeExports(); + makeMachine(); + + close_ragel_def(); +} + +void InputData::writeLanguage( std::ostream &out ) +{ + out << " lang=\""; + switch ( hostLang->lang ) { + case HostLang::C: out << "C"; break; + case HostLang::D: out << "D"; break; + case HostLang::D2: out << "D2"; break; + case HostLang::Go: out << "Go"; break; + case HostLang::Java: out << "Java"; break; + case HostLang::Ruby: out << "Ruby"; break; + case HostLang::CSharp: out << "C#"; break; + case HostLang::OCaml: out << "OCaml"; break; + } + out << "\""; +} + +void InputData::writeXML( std::ostream &out ) +{ + out << "<ragel version=\"" VERSION "\" filename=\"" << inputFileName << "\""; + writeLanguage( out ); + out << ">\n"; + + for ( ParserDict::Iter parser = parserDict; parser.lte(); parser++ ) { + ParseData *pd = parser->value->pd; + if ( pd->instanceList.length() > 0 ) + pd->generateXML( *outStream ); + } + + out << "</ragel>\n"; +} diff --git a/contrib/tools/ragel6/xmlcodegen.h b/contrib/tools/ragel6/xmlcodegen.h new file mode 100644 index 0000000000..d20d6ea6e0 --- /dev/null +++ b/contrib/tools/ragel6/xmlcodegen.h @@ -0,0 +1,192 @@ +/* + * Copyright 2005-2007 Adrian Thurston <thurston@complang.org> + */ + +/* This file is part of Ragel. + * + * Ragel is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * Ragel is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with Ragel; if not, write to the Free Software + * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + */ + +#ifndef _XMLCODEGEN_H +#define _XMLCODEGEN_H + +#include <iostream> +#include "avltree.h" +#include "fsmgraph.h" +#include "parsedata.h" +#include "redfsm.h" + +/* Forwards. */ +struct TransAp; +struct FsmAp; +struct ParseData; +struct GenInlineList; +struct CodeGenData; + +struct RedActionTable +: + public AvlTreeEl<RedActionTable> +{ + RedActionTable( const ActionTable &key ) + : + key(key), + id(0) + { } + + const ActionTable &getKey() + { return key; } + + ActionTable key; + int id; +}; + +typedef AvlTree<RedActionTable, ActionTable, CmpActionTable> ActionTableMap; + +struct NextRedTrans +{ + Key lowKey, highKey; + TransAp *trans; + TransAp *next; + + void load() { + if ( trans != 0 ) { + next = trans->next; + lowKey = trans->lowKey; + highKey = trans->highKey; + } + } + + NextRedTrans( TransAp *t ) { + trans = t; + load(); + } + + void increment() { + trans = next; + load(); + } +}; + +struct GenBase +{ + GenBase( char *fsmName, ParseData *pd, FsmAp *fsm ); + + void appendTrans( TransListVect &outList, Key lowKey, Key highKey, TransAp *trans ); + void reduceActionTables(); + + char *fsmName; + ParseData *pd; + FsmAp *fsm; + + ActionTableMap actionTableMap; + int nextActionTableId; +}; + +class XMLCodeGen : protected GenBase +{ +public: + XMLCodeGen( char *fsmName, ParseData *pd, FsmAp *fsm, std::ostream &out ); + + void writeXML( ); + +private: + void writeStateActions( StateAp *state ); + void writeStateList(); + void writeStateConditions( StateAp *state ); + + void writeKey( Key key ); + void writeText( InlineItem *item ); + void writeGoto( InlineItem *item ); + void writeGotoExpr( InlineItem *item ); + void writeCall( InlineItem *item ); + void writeCallExpr( InlineItem *item ); + void writeNext( InlineItem *item ); + void writeNextExpr( InlineItem *item ); + void writeEntry( InlineItem *item ); + void writeLmOnLast( InlineItem *item ); + void writeLmOnNext( InlineItem *item ); + void writeLmOnLagBehind( InlineItem *item ); + + void writeExports(); + bool writeNameInst( NameInst *nameInst ); + void writeEntryPoints(); + void writeConditions(); + void writeInlineList( InlineList *inlineList ); + void writeActionList(); + void writeActionTableList(); + void reduceTrans( TransAp *trans ); + void writeTransList( StateAp *state ); + void writeEofTrans( StateAp *state ); + void writeTrans( Key lowKey, Key highKey, TransAp *defTrans ); + void writeAction( Action *action ); + void writeLmSwitch( InlineItem *item ); + void writeMachine(); + void writeActionExec( InlineItem *item ); + + std::ostream &out; +}; + +class BackendGen : protected GenBase +{ +public: + BackendGen( char *fsmName, ParseData *pd, FsmAp *fsm, CodeGenData *cgd ); + void makeBackend( ); + +private: + void makeGenInlineList( GenInlineList *outList, InlineList *inList ); + void makeKey( GenInlineList *outList, Key key ); + void makeText( GenInlineList *outList, InlineItem *item ); + void makeLmOnLast( GenInlineList *outList, InlineItem *item ); + void makeLmOnNext( GenInlineList *outList, InlineItem *item ); + void makeLmOnLagBehind( GenInlineList *outList, InlineItem *item ); + void makeActionExec( GenInlineList *outList, InlineItem *item ); + void makeLmSwitch( GenInlineList *outList, InlineItem *item ); + void makeSetTokend( GenInlineList *outList, long offset ); + void makeSetAct( GenInlineList *outList, long lmId ); + void makeSubList( GenInlineList *outList, InlineList *inlineList, + GenInlineItem::Type type ); + void makeTargetItem( GenInlineList *outList, NameInst *nameTarg, GenInlineItem::Type type ); + void makeExecGetTokend( GenInlineList *outList ); + void makeExports(); + void makeMachine(); + void makeActionList(); + void makeAction( Action *action ); + void makeActionTableList(); + void makeConditions(); + void makeEntryPoints(); + bool makeNameInst( std::string &out, NameInst *nameInst ); + void makeStateList(); + + void makeStateActions( StateAp *state ); + void makeEofTrans( StateAp *state ); + void makeStateConditions( StateAp *state ); + void makeTransList( StateAp *state ); + void makeTrans( Key lowKey, Key highKey, TransAp *trans ); + + void close_ragel_def(); + + CodeGenData *cgd; + + /* Collected during parsing. */ + int curAction; + int curActionTable; + int curTrans; + int curState; + int curCondSpace; + int curStateCond; + +}; + +#endif diff --git a/contrib/tools/ragel6/ya.make b/contrib/tools/ragel6/ya.make new file mode 100644 index 0000000000..2c97f636ae --- /dev/null +++ b/contrib/tools/ragel6/ya.make @@ -0,0 +1,15 @@ +# Generated by devtools/yamaker from nixpkgs 980c4c3c2f664ccc5002f7fd6e08059cf1f00e75. + +OWNER(pg g:cpp-contrib) + +IF (USE_PREBUILT_TOOLS) + INCLUDE(${ARCADIA_ROOT}/build/prebuilt/contrib/tools/ragel6/ya.make.prebuilt) +ENDIF() + +IF (NOT PREBUILT) + INCLUDE(${ARCADIA_ROOT}/contrib/tools/ragel6/bin/ya.make) +ENDIF() + +RECURSE( + bin +) |