|
1
|
1 <html>
|
|
|
2 <head>
|
|
|
3 <title>MayaChemTools:Documentation:Parsers::YYLexer.pm</title>
|
|
|
4 <meta http-equiv="content-type" content="text/html;charset=utf-8">
|
|
|
5 <link rel="stylesheet" type="text/css" href="../../css/MayaChemTools.css">
|
|
|
6 </head>
|
|
|
7 <body leftmargin="20" rightmargin="20" topmargin="10" bottommargin="10">
|
|
|
8 <br/>
|
|
|
9 <center>
|
|
|
10 <a href="http://www.mayachemtools.org" title="MayaChemTools Home"><img src="../../images/MayaChemToolsLogo.gif" border="0" alt="MayaChemTools"></a>
|
|
|
11 </center>
|
|
|
12 <br/>
|
|
|
13 <div class="DocNav">
|
|
|
14 <table width="100%" border=0 cellpadding=0 cellspacing=2>
|
|
|
15 <tr align="left" valign="top"><td width="33%" align="left"><a href="./SimpleCalcYYLexer.html" title="SimpleCalcYYLexer.html">Previous</a> <a href="./index.html" title="Table of Contents">TOC</a> <a href="./PDBFileUtil.html" title="PDBFileUtil.html">Next</a></td><td width="34%" align="middle"><strong>Parsers::YYLexer.pm</strong></td><td width="33%" align="right"><a href="././code/YYLexer.html" title="View source code">Code</a> | <a href="./../pdf/YYLexer.pdf" title="PDF US Letter Size">PDF</a> | <a href="./../pdfgreen/YYLexer.pdf" title="PDF US Letter Size with narrow margins: www.changethemargins.com">PDFGreen</a> | <a href="./../pdfa4/YYLexer.pdf" title="PDF A4 Size">PDFA4</a> | <a href="./../pdfa4green/YYLexer.pdf" title="PDF A4 Size with narrow margins: www.changethemargins.com">PDFA4Green</a></td></tr>
|
|
|
16 </table>
|
|
|
17 </div>
|
|
|
18 <p>
|
|
|
19 </p>
|
|
|
20 <h2>NAME</h2>
|
|
|
21 <p>Parsers::YYLexer</p>
|
|
|
22 <p>
|
|
|
23 </p>
|
|
|
24 <h2>SYNOPSIS</h2>
|
|
|
25 <p>use Parseres::YYLexer;</p>
|
|
|
26 <p>use Parsers::YYLexer qw(:all);</p>
|
|
|
27 <p>
|
|
|
28 </p>
|
|
|
29 <h2>DESCRIPTION</h2>
|
|
|
30 <p><strong>YYLexer</strong> class provides the following methods:</p>
|
|
|
31 <p> <a href="#new">new</a>, <a href="#getyylex">GetYYLex</a>, <a href="#next">Next</a>, <a href="#peek">Peek</a>, <a href="#setupyytabfile">SetupYYTabFile</a>, <a href="#stringifyyylexer">StringifyYYLexer</a>, <a href="#yylex">YYLex</a>
|
|
|
32 </p><p><strong>Parsers::YYLexer</strong> class is derived from <strong>Parsers::Lexer</strong> base class, which provides all
|
|
|
33 the underlying lexer functionality. <strong>YYLexer</strong> class is designed to be used with
|
|
|
34 <strong>yyparse</strong> code generated by running <strong>byacc</strong> on a parsers defined using
|
|
|
35 parser definition <strong>ParserName.yy</strong> file.</p>
|
|
|
36 <p><em>YYTabFile</em> containing mapping of token labels to integers must be explicitly
|
|
|
37 specified by the caller. This file is processed during new method invocation and
|
|
|
38 mapping of token labels to integers is loaded in a hash to be used later by <strong>YYLex</strong>
|
|
|
39 method to return token number and text pairs to the parser.</p>
|
|
|
40 <p>
|
|
|
41 </p>
|
|
|
42 <h2>METHODS</h2>
|
|
|
43 <dl>
|
|
|
44 <dt><strong><a name="new" class="item"><strong>new</strong></a></strong></dt>
|
|
|
45 <dd>
|
|
|
46 <div class="OptionsBox">
|
|
|
47 $YYLexer = new Parsers::YYLexer($Input, @YYLexerTokensSpec);</div>
|
|
|
48 <p>Using specified <em>Input</em> and <em>YYLexerTokensSpec</em>, <strong>new</strong> method generates a new
|
|
|
49 <strong>YYLexer</strong> and returns a reference to newly created <strong>YYLexer</strong> object.</p>
|
|
|
50 <p>Examples:</p>
|
|
|
51 <div class="OptionsBox">
|
|
|
52 # Tokens specifications supplied by the caller. It's an array containing references
|
|
|
53 <br/> # to arrays with each containing TokenLabel and TokenMatchRegex pair along with
|
|
|
54 <br/> # an option reference to code to be executed after a matched.
|
|
|
55 <br/> #
|
|
|
56 <br/> @LexerTokensSpec = (
|
|
|
57 <br/> [ 'LETTER', qr/[a-zA-Z]/ ],
|
|
|
58 [ 'NUMBER', qr/\d+/ ],
|
|
|
59 [ 'SPACE', qr/[ ]*/,
|
|
|
60 <br/> sub { my($This, $TokenLabel, $MatchedText) = @_; return ''; }
|
|
|
61 <br/> ],
|
|
|
62 [ 'NEWLINE', qr/(?:\r\n|\r|\n)/,
|
|
|
63 <br/> sub { my($This, $TokenLabel, $MatchedText) = @_; return "\n"; }
|
|
|
64 <br/> ],
|
|
|
65 [ 'CHAR', qr/./ ]
|
|
|
66 <br/> );</div>
|
|
|
67 <div class="OptionsBox">
|
|
|
68 # Input string...
|
|
|
69 <br/> $InputText = 'y = 3 + 4';</div>
|
|
|
70 <div class="OptionsBox">
|
|
|
71 $YLexer = new Parsers::YYLexer($InputText, @LexerTokensSpec);</div>
|
|
|
72 <div class="OptionsBox">
|
|
|
73 # Setup default token table file...
|
|
|
74 <br/> $YYTabFile = "Parsers/SimpleCalcParser.tab.ph";
|
|
|
75 <br/> $This->SetupYYTabFile($YYTabFile);</div>
|
|
|
76 <div class="OptionsBox">
|
|
|
77 # Process input stream...
|
|
|
78 <br/> ($TokenNumber, $TokenText) = $YYLexer->Lex();
|
|
|
79 <br/> print "TokenNumber: $TokenNumber; TokenText: $TokenText\n";</div>
|
|
|
80 <div class="OptionsBox">
|
|
|
81 # Input file...
|
|
|
82 <br/> $InputFile = "Input.txt";
|
|
|
83 <br/> open INPUTFILE, "$InputFile" or die "Couldn't open $InputFile: $!\n";
|
|
|
84 <br/> $Lexer = new Parsers::YYLexer(\*INPUTFILE, @LexerTokensSpec);</div>
|
|
|
85 <div class="OptionsBox">
|
|
|
86 # Input file iterator...
|
|
|
87 <br/> $InputFile = "TestSimpleCalcParser.txt";
|
|
|
88 <br/> open INPUTFILE, "$InputFile" or die "Couldn't open $InputFile: $!\n";
|
|
|
89 <br/> $InputIterator = sub { return <INPUTFILE>; };
|
|
|
90 <br/> $Lexer = new Parsers::YYLexer($InputIterator, @LexerTokensSpec);</div>
|
|
|
91 <div class="OptionsBox">
|
|
|
92 # Usage with code generated by byacc from a parser definition
|
|
|
93 <br/> # file SimpleCalcParser.yy...</div>
|
|
|
94 <div class="OptionsBox">
|
|
|
95 $InputText = "3 + 4 +6\nx=3\ny=5\nx+y\nx+z\n";</div>
|
|
|
96 <div class="OptionsBox">
|
|
|
97 $YYLexer = new Parsers::YYLexer($InputText,@LexerTokensSpec);</div>
|
|
|
98 <div class="OptionsBox">
|
|
|
99 $YYLex = $YYLexer->GetYYLex();</div>
|
|
|
100 <div class="OptionsBox">
|
|
|
101 $YYTabFile = "Parsers/SimpleCalcParser.tab.ph";
|
|
|
102 <br/> $YYLexer->SetupYYTabFile($YYTabFile);</div>
|
|
|
103 <div class="OptionsBox">
|
|
|
104 $Debug = 0;
|
|
|
105 <br/> $SimpleCalcParser = new Parsers::SimpleCalcParser($YYLex,
|
|
|
106 \&Parsers::SimpleCalcParser::yyerror, $Debug);</div>
|
|
|
107 <div class="OptionsBox">
|
|
|
108 $Value = $SimpleCalcParser->yyparse();
|
|
|
109 <br/> print "Value = " . (defined($Value) ? "$Value" : "Undefined") . "\n";</div>
|
|
|
110 </dd>
|
|
|
111 <dt><strong><a name="getyylex" class="item"><strong>GetYYLex</strong></a></strong></dt>
|
|
|
112 <dd>
|
|
|
113 <div class="OptionsBox">
|
|
|
114 $YYLex = $YYLexer->GetYYLex();</div>
|
|
|
115 <p>Returns a curried verson of YYLexer as <strong>YYLex</strong>: yyparse in parser generated by
|
|
|
116 byacc expects it to call without passing any argument for the <em>YYLexer</em> object.</p>
|
|
|
117 </dd>
|
|
|
118 <dt><strong><a name="next" class="item"><strong>Next</strong></a></strong></dt>
|
|
|
119 <dd>
|
|
|
120 <div class="OptionsBox">
|
|
|
121 ($TokenNumber, $TokenText) = $YYLexer->Next();</div>
|
|
|
122 <p>Returns next available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from
|
|
|
123 input stream by removing it from the input stream. Token number and text of
|
|
|
124 zero corresponds to end of input (EOI).</p>
|
|
|
125 </dd>
|
|
|
126 <dt><strong><a name="peek" class="item"><strong>Peek</strong></a></strong></dt>
|
|
|
127 <dd>
|
|
|
128 <div class="OptionsBox">
|
|
|
129 ($TokenNumber, $TokenText) = $YYLexer->Peek();</div>
|
|
|
130 <p>Returns next available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from
|
|
|
131 input stream by simply looking ahead and without removing it from the input stream.
|
|
|
132 Token number and text of zero corresponds to end of input (EOI).</p>
|
|
|
133 </dd>
|
|
|
134 <dt><strong><a name="setupyytabfile" class="item"><strong>SetupYYTabFile</strong></a></strong></dt>
|
|
|
135 <dd>
|
|
|
136 <div class="OptionsBox">
|
|
|
137 $YYLexer = $YYLexer->SetupYYTabFile($YYTabFile);</div>
|
|
|
138 <p>Processes token labels to integers data map in specified <em>YYTabFile</em> and returns
|
|
|
139 <em>YYLexer</em>.</p>
|
|
|
140 <p>Notes:</p>
|
|
|
141 <div class="OptionsBox">
|
|
|
142 . YYTabFile must be a complete path or available through @INC path in the
|
|
|
143 same directory where this package is located.
|
|
|
144 <br/> . Name of YYTabFile might start with any valid sub directory name in @INC
|
|
|
145 For example, "Parsers/<YYTablFile>" implies the tab file in parsers sub
|
|
|
146 directory under MayaChemTools lib directory as it would be already in @INC
|
|
|
147 path.
|
|
|
148 <br/> . YYTabFile must be explicitly set by the caller. The default YYTabFile name,
|
|
|
149 y.tab.ph, generated by byacc is not used implicitly to avoid confusion
|
|
|
150 among multiple distinct instances of YYLexer.
|
|
|
151 <br/> . YYTabFile is generated by byacc during its usage with -d options and
|
|
|
152 contains mapping of token codes to token names/labels. YYLexer used this
|
|
|
153 file to map token labels to token codes before returning token code and
|
|
|
154 value pair back to yyparse function used by byacc.
|
|
|
155 <br/> . User defined token numbers start from 257
|
|
|
156 <br/> . Token number for any user defined token EOI is mapped to its value before
|
|
|
157 default token number of 0 for EOI.</div>
|
|
|
158 <div class="OptionsBox">
|
|
|
159 The format of YYTabFile generated by byacc during generation of parser code in
|
|
|
160 <br/> Perl code is:</div>
|
|
|
161 <div class="OptionsBox">
|
|
|
162 ... ...
|
|
|
163 <br/> $NUMBER=257;
|
|
|
164 <br/> $ADDOP=258;
|
|
|
165 <br/> $SUBOP=259;
|
|
|
166 <br/> ... ..</div>
|
|
|
167 </dd>
|
|
|
168 <dt><strong><a name="yylex" class="item"><strong>YYLex</strong></a></strong></dt>
|
|
|
169 <dd>
|
|
|
170 <div class="OptionsBox">
|
|
|
171 ($TokenNumber, $TokenText) = $YYLexer->YYLex();
|
|
|
172 <br/> ($TokenNumber, $TokenText) = $YYLexer->YYLex($Mode);</div>
|
|
|
173 <p>Returns available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from
|
|
|
174 input stream by either removing it from the input stream or by simply looking
|
|
|
175 ahead and without removing it from the input stream. Token number and text of
|
|
|
176 zero corresponds to end of input (EOI).</p>
|
|
|
177 <p>Possible <em>Mode</em> values: <em>Peek, Next</em>. Default: <em>Next</em>.</p>
|
|
|
178 <p><em>YYLex</em> is designed to be used with <strong>yyparse</strong> code generated by running
|
|
|
179 <strong>byacc</strong> on a parsers defined using parser definition <strong>ParserName.yy</strong> file.</p>
|
|
|
180 <p>Notes:</p>
|
|
|
181 <div class="OptionsBox">
|
|
|
182 . Token label and value pairs returned by Lexer from by base class, which
|
|
|
183 can't be mapped to token labels specified in YYTabFile are ignored.
|
|
|
184 <br/> . Token text of length 1 returned by Lexer from base class without a
|
|
|
185 corresponding explicit token label, which can't be mapped to a token
|
|
|
186 number using Perl ord function, is ignored.</div>
|
|
|
187 </dd>
|
|
|
188 <dt><strong><a name="stringifyyylexer" class="item"><strong>StringifyYYLexer</strong></a></strong></dt>
|
|
|
189 <dd>
|
|
|
190 <div class="OptionsBox">
|
|
|
191 $YYLexerString = $YYLexer->StringifyYYLexer();</div>
|
|
|
192 <p>Returns a string containing information about <em>YYLexer</em> object.</p>
|
|
|
193 </dd>
|
|
|
194 </dl>
|
|
|
195 <p>
|
|
|
196 </p>
|
|
|
197 <h2>AUTHOR</h2>
|
|
|
198 <p><a href="mailto:msud@san.rr.com">Manish Sud</a></p>
|
|
|
199 <p>
|
|
|
200 </p>
|
|
|
201 <h2>SEE ALSO</h2>
|
|
|
202 <p><a href="./Lexer.html">Lexer.pm</a>, <a href="./SimpleCalcYYLexer.html">SimpleCalcYYLexer.pm</a>, <a href="./SimpleCalcParser.html">SimpleCalcParser.yy</a>
|
|
|
203 </p>
|
|
|
204 <p>
|
|
|
205 </p>
|
|
|
206 <h2>COPYRIGHT</h2>
|
|
|
207 <p>Copyright (C) 2015 Manish Sud. All rights reserved.</p>
|
|
|
208 <p>This file is part of MayaChemTools.</p>
|
|
|
209 <p>MayaChemTools is free software; you can redistribute it and/or modify it under
|
|
|
210 the terms of the GNU Lesser General Public License as published by the Free
|
|
|
211 Software Foundation; either version 3 of the License, or (at your option)
|
|
|
212 any later version.</p>
|
|
|
213 <p> </p><p> </p><div class="DocNav">
|
|
|
214 <table width="100%" border=0 cellpadding=0 cellspacing=2>
|
|
|
215 <tr align="left" valign="top"><td width="33%" align="left"><a href="./SimpleCalcYYLexer.html" title="SimpleCalcYYLexer.html">Previous</a> <a href="./index.html" title="Table of Contents">TOC</a> <a href="./PDBFileUtil.html" title="PDBFileUtil.html">Next</a></td><td width="34%" align="middle"><strong>March 29, 2015</strong></td><td width="33%" align="right"><strong>Parsers::YYLexer.pm</strong></td></tr>
|
|
|
216 </table>
|
|
|
217 </div>
|
|
|
218 <br />
|
|
|
219 <center>
|
|
|
220 <img src="../../images/h2o2.png">
|
|
|
221 </center>
|
|
|
222 </body>
|
|
|
223 </html>
|
