Mercurial > repos > deepakjadmin > mayatool3_test3
comparison mayachemtools/docs/modules/html/YYLexer.html @ 0:73ae111cf86f draft
Uploaded
author | deepakjadmin |
---|---|
date | Wed, 20 Jan 2016 11:55:01 -0500 |
parents | |
children |
comparison
equal
deleted
inserted
replaced
-1:000000000000 | 0:73ae111cf86f |
---|---|
1 <html> | |
2 <head> | |
3 <title>MayaChemTools:Documentation:Parsers::YYLexer.pm</title> | |
4 <meta http-equiv="content-type" content="text/html;charset=utf-8"> | |
5 <link rel="stylesheet" type="text/css" href="../../css/MayaChemTools.css"> | |
6 </head> | |
7 <body leftmargin="20" rightmargin="20" topmargin="10" bottommargin="10"> | |
8 <br/> | |
9 <center> | |
10 <a href="http://www.mayachemtools.org" title="MayaChemTools Home"><img src="../../images/MayaChemToolsLogo.gif" border="0" alt="MayaChemTools"></a> | |
11 </center> | |
12 <br/> | |
13 <div class="DocNav"> | |
14 <table width="100%" border=0 cellpadding=0 cellspacing=2> | |
15 <tr align="left" valign="top"><td width="33%" align="left"><a href="./SimpleCalcYYLexer.html" title="SimpleCalcYYLexer.html">Previous</a> <a href="./index.html" title="Table of Contents">TOC</a> <a href="./PDBFileUtil.html" title="PDBFileUtil.html">Next</a></td><td width="34%" align="middle"><strong>Parsers::YYLexer.pm</strong></td><td width="33%" align="right"><a href="././code/YYLexer.html" title="View source code">Code</a> | <a href="./../pdf/YYLexer.pdf" title="PDF US Letter Size">PDF</a> | <a href="./../pdfgreen/YYLexer.pdf" title="PDF US Letter Size with narrow margins: www.changethemargins.com">PDFGreen</a> | <a href="./../pdfa4/YYLexer.pdf" title="PDF A4 Size">PDFA4</a> | <a href="./../pdfa4green/YYLexer.pdf" title="PDF A4 Size with narrow margins: www.changethemargins.com">PDFA4Green</a></td></tr> | |
16 </table> | |
17 </div> | |
18 <p> | |
19 </p> | |
20 <h2>NAME</h2> | |
21 <p>Parsers::YYLexer</p> | |
22 <p> | |
23 </p> | |
24 <h2>SYNOPSIS</h2> | |
25 <p>use Parseres::YYLexer;</p> | |
26 <p>use Parsers::YYLexer qw(:all);</p> | |
27 <p> | |
28 </p> | |
29 <h2>DESCRIPTION</h2> | |
30 <p><strong>YYLexer</strong> class provides the following methods:</p> | |
31 <p> <a href="#new">new</a>, <a href="#getyylex">GetYYLex</a>, <a href="#next">Next</a>, <a href="#peek">Peek</a>, <a href="#setupyytabfile">SetupYYTabFile</a>, <a href="#stringifyyylexer">StringifyYYLexer</a>, <a href="#yylex">YYLex</a> | |
32 </p><p><strong>Parsers::YYLexer</strong> class is derived from <strong>Parsers::Lexer</strong> base class, which provides all | |
33 the underlying lexer functionality. <strong>YYLexer</strong> class is designed to be used with | |
34 <strong>yyparse</strong> code generated by running <strong>byacc</strong> on a parsers defined using | |
35 parser definition <strong>ParserName.yy</strong> file.</p> | |
36 <p><em>YYTabFile</em> containing mapping of token labels to integers must be explicitly | |
37 specified by the caller. This file is processed during new method invocation and | |
38 mapping of token labels to integers is loaded in a hash to be used later by <strong>YYLex</strong> | |
39 method to return token number and text pairs to the parser.</p> | |
40 <p> | |
41 </p> | |
42 <h2>METHODS</h2> | |
43 <dl> | |
44 <dt><strong><a name="new" class="item"><strong>new</strong></a></strong></dt> | |
45 <dd> | |
46 <div class="OptionsBox"> | |
47 $YYLexer = new Parsers::YYLexer($Input, @YYLexerTokensSpec);</div> | |
48 <p>Using specified <em>Input</em> and <em>YYLexerTokensSpec</em>, <strong>new</strong> method generates a new | |
49 <strong>YYLexer</strong> and returns a reference to newly created <strong>YYLexer</strong> object.</p> | |
50 <p>Examples:</p> | |
51 <div class="OptionsBox"> | |
52 # Tokens specifications supplied by the caller. It's an array containing references | |
53 <br/> # to arrays with each containing TokenLabel and TokenMatchRegex pair along with | |
54 <br/> # an option reference to code to be executed after a matched. | |
55 <br/> # | |
56 <br/> @LexerTokensSpec = ( | |
57 <br/> [ 'LETTER', qr/[a-zA-Z]/ ], | |
58 [ 'NUMBER', qr/\d+/ ], | |
59 [ 'SPACE', qr/[ ]*/, | |
60 <br/> sub { my($This, $TokenLabel, $MatchedText) = @_; return ''; } | |
61 <br/> ], | |
62 [ 'NEWLINE', qr/(?:\r\n|\r|\n)/, | |
63 <br/> sub { my($This, $TokenLabel, $MatchedText) = @_; return "\n"; } | |
64 <br/> ], | |
65 [ 'CHAR', qr/./ ] | |
66 <br/> );</div> | |
67 <div class="OptionsBox"> | |
68 # Input string... | |
69 <br/> $InputText = 'y = 3 + 4';</div> | |
70 <div class="OptionsBox"> | |
71 $YLexer = new Parsers::YYLexer($InputText, @LexerTokensSpec);</div> | |
72 <div class="OptionsBox"> | |
73 # Setup default token table file... | |
74 <br/> $YYTabFile = "Parsers/SimpleCalcParser.tab.ph"; | |
75 <br/> $This->SetupYYTabFile($YYTabFile);</div> | |
76 <div class="OptionsBox"> | |
77 # Process input stream... | |
78 <br/> ($TokenNumber, $TokenText) = $YYLexer->Lex(); | |
79 <br/> print "TokenNumber: $TokenNumber; TokenText: $TokenText\n";</div> | |
80 <div class="OptionsBox"> | |
81 # Input file... | |
82 <br/> $InputFile = "Input.txt"; | |
83 <br/> open INPUTFILE, "$InputFile" or die "Couldn't open $InputFile: $!\n"; | |
84 <br/> $Lexer = new Parsers::YYLexer(\*INPUTFILE, @LexerTokensSpec);</div> | |
85 <div class="OptionsBox"> | |
86 # Input file iterator... | |
87 <br/> $InputFile = "TestSimpleCalcParser.txt"; | |
88 <br/> open INPUTFILE, "$InputFile" or die "Couldn't open $InputFile: $!\n"; | |
89 <br/> $InputIterator = sub { return <INPUTFILE>; }; | |
90 <br/> $Lexer = new Parsers::YYLexer($InputIterator, @LexerTokensSpec);</div> | |
91 <div class="OptionsBox"> | |
92 # Usage with code generated by byacc from a parser definition | |
93 <br/> # file SimpleCalcParser.yy...</div> | |
94 <div class="OptionsBox"> | |
95 $InputText = "3 + 4 +6\nx=3\ny=5\nx+y\nx+z\n";</div> | |
96 <div class="OptionsBox"> | |
97 $YYLexer = new Parsers::YYLexer($InputText,@LexerTokensSpec);</div> | |
98 <div class="OptionsBox"> | |
99 $YYLex = $YYLexer->GetYYLex();</div> | |
100 <div class="OptionsBox"> | |
101 $YYTabFile = "Parsers/SimpleCalcParser.tab.ph"; | |
102 <br/> $YYLexer->SetupYYTabFile($YYTabFile);</div> | |
103 <div class="OptionsBox"> | |
104 $Debug = 0; | |
105 <br/> $SimpleCalcParser = new Parsers::SimpleCalcParser($YYLex, | |
106 \&Parsers::SimpleCalcParser::yyerror, $Debug);</div> | |
107 <div class="OptionsBox"> | |
108 $Value = $SimpleCalcParser->yyparse(); | |
109 <br/> print "Value = " . (defined($Value) ? "$Value" : "Undefined") . "\n";</div> | |
110 </dd> | |
111 <dt><strong><a name="getyylex" class="item"><strong>GetYYLex</strong></a></strong></dt> | |
112 <dd> | |
113 <div class="OptionsBox"> | |
114 $YYLex = $YYLexer->GetYYLex();</div> | |
115 <p>Returns a curried verson of YYLexer as <strong>YYLex</strong>: yyparse in parser generated by | |
116 byacc expects it to call without passing any argument for the <em>YYLexer</em> object.</p> | |
117 </dd> | |
118 <dt><strong><a name="next" class="item"><strong>Next</strong></a></strong></dt> | |
119 <dd> | |
120 <div class="OptionsBox"> | |
121 ($TokenNumber, $TokenText) = $YYLexer->Next();</div> | |
122 <p>Returns next available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from | |
123 input stream by removing it from the input stream. Token number and text of | |
124 zero corresponds to end of input (EOI).</p> | |
125 </dd> | |
126 <dt><strong><a name="peek" class="item"><strong>Peek</strong></a></strong></dt> | |
127 <dd> | |
128 <div class="OptionsBox"> | |
129 ($TokenNumber, $TokenText) = $YYLexer->Peek();</div> | |
130 <p>Returns next available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from | |
131 input stream by simply looking ahead and without removing it from the input stream. | |
132 Token number and text of zero corresponds to end of input (EOI).</p> | |
133 </dd> | |
134 <dt><strong><a name="setupyytabfile" class="item"><strong>SetupYYTabFile</strong></a></strong></dt> | |
135 <dd> | |
136 <div class="OptionsBox"> | |
137 $YYLexer = $YYLexer->SetupYYTabFile($YYTabFile);</div> | |
138 <p>Processes token labels to integers data map in specified <em>YYTabFile</em> and returns | |
139 <em>YYLexer</em>.</p> | |
140 <p>Notes:</p> | |
141 <div class="OptionsBox"> | |
142 . YYTabFile must be a complete path or available through @INC path in the | |
143 same directory where this package is located. | |
144 <br/> . Name of YYTabFile might start with any valid sub directory name in @INC | |
145 For example, "Parsers/<YYTablFile>" implies the tab file in parsers sub | |
146 directory under MayaChemTools lib directory as it would be already in @INC | |
147 path. | |
148 <br/> . YYTabFile must be explicitly set by the caller. The default YYTabFile name, | |
149 y.tab.ph, generated by byacc is not used implicitly to avoid confusion | |
150 among multiple distinct instances of YYLexer. | |
151 <br/> . YYTabFile is generated by byacc during its usage with -d options and | |
152 contains mapping of token codes to token names/labels. YYLexer used this | |
153 file to map token labels to token codes before returning token code and | |
154 value pair back to yyparse function used by byacc. | |
155 <br/> . User defined token numbers start from 257 | |
156 <br/> . Token number for any user defined token EOI is mapped to its value before | |
157 default token number of 0 for EOI.</div> | |
158 <div class="OptionsBox"> | |
159 The format of YYTabFile generated by byacc during generation of parser code in | |
160 <br/> Perl code is:</div> | |
161 <div class="OptionsBox"> | |
162 ... ... | |
163 <br/> $NUMBER=257; | |
164 <br/> $ADDOP=258; | |
165 <br/> $SUBOP=259; | |
166 <br/> ... ..</div> | |
167 </dd> | |
168 <dt><strong><a name="yylex" class="item"><strong>YYLex</strong></a></strong></dt> | |
169 <dd> | |
170 <div class="OptionsBox"> | |
171 ($TokenNumber, $TokenText) = $YYLexer->YYLex(); | |
172 <br/> ($TokenNumber, $TokenText) = $YYLexer->YYLex($Mode);</div> | |
173 <p>Returns available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from | |
174 input stream by either removing it from the input stream or by simply looking | |
175 ahead and without removing it from the input stream. Token number and text of | |
176 zero corresponds to end of input (EOI).</p> | |
177 <p>Possible <em>Mode</em> values: <em>Peek, Next</em>. Default: <em>Next</em>.</p> | |
178 <p><em>YYLex</em> is designed to be used with <strong>yyparse</strong> code generated by running | |
179 <strong>byacc</strong> on a parsers defined using parser definition <strong>ParserName.yy</strong> file.</p> | |
180 <p>Notes:</p> | |
181 <div class="OptionsBox"> | |
182 . Token label and value pairs returned by Lexer from by base class, which | |
183 can't be mapped to token labels specified in YYTabFile are ignored. | |
184 <br/> . Token text of length 1 returned by Lexer from base class without a | |
185 corresponding explicit token label, which can't be mapped to a token | |
186 number using Perl ord function, is ignored.</div> | |
187 </dd> | |
188 <dt><strong><a name="stringifyyylexer" class="item"><strong>StringifyYYLexer</strong></a></strong></dt> | |
189 <dd> | |
190 <div class="OptionsBox"> | |
191 $YYLexerString = $YYLexer->StringifyYYLexer();</div> | |
192 <p>Returns a string containing information about <em>YYLexer</em> object.</p> | |
193 </dd> | |
194 </dl> | |
195 <p> | |
196 </p> | |
197 <h2>AUTHOR</h2> | |
198 <p><a href="mailto:msud@san.rr.com">Manish Sud</a></p> | |
199 <p> | |
200 </p> | |
201 <h2>SEE ALSO</h2> | |
202 <p><a href="./Lexer.html">Lexer.pm</a>, <a href="./SimpleCalcYYLexer.html">SimpleCalcYYLexer.pm</a>, <a href="./SimpleCalcParser.html">SimpleCalcParser.yy</a> | |
203 </p> | |
204 <p> | |
205 </p> | |
206 <h2>COPYRIGHT</h2> | |
207 <p>Copyright (C) 2015 Manish Sud. All rights reserved.</p> | |
208 <p>This file is part of MayaChemTools.</p> | |
209 <p>MayaChemTools is free software; you can redistribute it and/or modify it under | |
210 the terms of the GNU Lesser General Public License as published by the Free | |
211 Software Foundation; either version 3 of the License, or (at your option) | |
212 any later version.</p> | |
213 <p> </p><p> </p><div class="DocNav"> | |
214 <table width="100%" border=0 cellpadding=0 cellspacing=2> | |
215 <tr align="left" valign="top"><td width="33%" align="left"><a href="./SimpleCalcYYLexer.html" title="SimpleCalcYYLexer.html">Previous</a> <a href="./index.html" title="Table of Contents">TOC</a> <a href="./PDBFileUtil.html" title="PDBFileUtil.html">Next</a></td><td width="34%" align="middle"><strong>March 29, 2015</strong></td><td width="33%" align="right"><strong>Parsers::YYLexer.pm</strong></td></tr> | |
216 </table> | |
217 </div> | |
218 <br /> | |
219 <center> | |
220 <img src="../../images/h2o2.png"> | |
221 </center> | |
222 </body> | |
223 </html> |