Mercurial > repos > deepakjadmin > mayatool3_test2

comparison docs/modules/html/YYLexer.html @ 0:4816e4a8ae95 draft default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Uploaded
author deepakjadmin
date Wed, 20 Jan 2016 09:23:18 -0500
parents
children
comparison
equal deleted inserted replaced
-1:000000000000 0:4816e4a8ae95
1 <html>
2 <head>
3 <title>MayaChemTools:Documentation:Parsers::YYLexer.pm</title>
4 <meta http-equiv="content-type" content="text/html;charset=utf-8">
5 <link rel="stylesheet" type="text/css" href="../../css/MayaChemTools.css">
6 </head>
7 <body leftmargin="20" rightmargin="20" topmargin="10" bottommargin="10">
8 <br/>
9 <center>
10 <a href="http://www.mayachemtools.org" title="MayaChemTools Home"><img src="../../images/MayaChemToolsLogo.gif" border="0" alt="MayaChemTools"></a>
11 </center>
12 <br/>
13 <div class="DocNav">
14 <table width="100%" border=0 cellpadding=0 cellspacing=2>
15 <tr align="left" valign="top"><td width="33%" align="left"><a href="./SimpleCalcYYLexer.html" title="SimpleCalcYYLexer.html">Previous</a>&nbsp;&nbsp;<a href="./index.html" title="Table of Contents">TOC</a>&nbsp;&nbsp;<a href="./PDBFileUtil.html" title="PDBFileUtil.html">Next</a></td><td width="34%" align="middle"><strong>Parsers::YYLexer.pm</strong></td><td width="33%" align="right"><a href="././code/YYLexer.html" title="View source code">Code</a>&nbsp;|&nbsp;<a href="./../pdf/YYLexer.pdf" title="PDF US Letter Size">PDF</a>&nbsp;|&nbsp;<a href="./../pdfgreen/YYLexer.pdf" title="PDF US Letter Size with narrow margins: www.changethemargins.com">PDFGreen</a>&nbsp;|&nbsp;<a href="./../pdfa4/YYLexer.pdf" title="PDF A4 Size">PDFA4</a>&nbsp;|&nbsp;<a href="./../pdfa4green/YYLexer.pdf" title="PDF A4 Size with narrow margins: www.changethemargins.com">PDFA4Green</a></td></tr>
16 </table>
17 </div>
18 <p>
19 </p>
20 <h2>NAME</h2>
21 <p>Parsers::YYLexer</p>
22 <p>
23 </p>
24 <h2>SYNOPSIS</h2>
25 <p>use Parseres::YYLexer;</p>
26 <p>use Parsers::YYLexer qw(:all);</p>
27 <p>
28 </p>
29 <h2>DESCRIPTION</h2>
30 <p><strong>YYLexer</strong> class provides the following methods:</p>
31 <p> <a href="#new">new</a>, <a href="#getyylex">GetYYLex</a>, <a href="#next">Next</a>, <a href="#peek">Peek</a>, <a href="#setupyytabfile">SetupYYTabFile</a>, <a href="#stringifyyylexer">StringifyYYLexer</a>, <a href="#yylex">YYLex</a>
32 </p><p><strong>Parsers::YYLexer</strong> class is derived from <strong>Parsers::Lexer</strong> base class, which provides all
33 the underlying lexer functionality. <strong>YYLexer</strong> class is designed to be used with
34 <strong>yyparse</strong> code generated by running <strong>byacc</strong> on a parsers defined using
35 parser definition <strong>ParserName.yy</strong> file.</p>
36 <p><em>YYTabFile</em> containing mapping of token labels to integers must be explicitly
37 specified by the caller. This file is processed during new method invocation and
38 mapping of token labels to integers is loaded in a hash to be used later by <strong>YYLex</strong>
39 method to return token number and text pairs to the parser.</p>
40 <p>
41 </p>
42 <h2>METHODS</h2>
43 <dl>
44 <dt><strong><a name="new" class="item"><strong>new</strong></a></strong></dt>
45 <dd>
46 <div class="OptionsBox">
47 $YYLexer = new Parsers::YYLexer($Input, @YYLexerTokensSpec);</div>
48 <p>Using specified <em>Input</em> and <em>YYLexerTokensSpec</em>, <strong>new</strong> method generates a new
49 <strong>YYLexer</strong> and returns a reference to newly created <strong>YYLexer</strong> object.</p>
50 <p>Examples:</p>
51 <div class="OptionsBox">
52 # Tokens specifications supplied by the caller. It's an array containing references
53 <br/> # to arrays with each containing TokenLabel and TokenMatchRegex pair along with
54 <br/> # an option reference to code to be executed after a matched.
55 <br/> #
56 <br/> @LexerTokensSpec = (
57 <br/>&nbsp;&nbsp;&nbsp;&nbsp; [ 'LETTER', qr/[a-zA-Z]/ ],
58 [ 'NUMBER', qr/\d+/ ],
59 [ 'SPACE', qr/[ ]*/,
60 <br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub { my($This, $TokenLabel, $MatchedText) = @_; return ''; }
61 <br/>&nbsp;&nbsp;&nbsp;&nbsp; ],
62 [ 'NEWLINE', qr/(?:\r\n|\r|\n)/,
63 <br/>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; sub { my($This, $TokenLabel, $MatchedText) = @_; return &quot;\n&quot;; }
64 <br/>&nbsp;&nbsp;&nbsp;&nbsp; ],
65 [ 'CHAR', qr/./ ]
66 <br/> );</div>
67 <div class="OptionsBox">
68 # Input string...
69 <br/> $InputText = 'y = 3 + 4';</div>
70 <div class="OptionsBox">
71 $YLexer = new Parsers::YYLexer($InputText, @LexerTokensSpec);</div>
72 <div class="OptionsBox">
73 # Setup default token table file...
74 <br/> $YYTabFile = &quot;Parsers/SimpleCalcParser.tab.ph&quot;;
75 <br/> $This-&gt;SetupYYTabFile($YYTabFile);</div>
76 <div class="OptionsBox">
77 # Process input stream...
78 <br/> ($TokenNumber, $TokenText) = $YYLexer-&gt;Lex();
79 <br/> print &quot;TokenNumber: $TokenNumber; TokenText: $TokenText\n&quot;;</div>
80 <div class="OptionsBox">
81 # Input file...
82 <br/> $InputFile = &quot;Input.txt&quot;;
83 <br/> open INPUTFILE, &quot;$InputFile&quot; or die &quot;Couldn't open $InputFile: $!\n&quot;;
84 <br/> $Lexer = new Parsers::YYLexer(\*INPUTFILE, @LexerTokensSpec);</div>
85 <div class="OptionsBox">
86 # Input file iterator...
87 <br/> $InputFile = &quot;TestSimpleCalcParser.txt&quot;;
88 <br/> open INPUTFILE, &quot;$InputFile&quot; or die &quot;Couldn't open $InputFile: $!\n&quot;;
89 <br/> $InputIterator = sub { return &lt;INPUTFILE&gt;; };
90 <br/> $Lexer = new Parsers::YYLexer($InputIterator, @LexerTokensSpec);</div>
91 <div class="OptionsBox">
92 # Usage with code generated by byacc from a parser definition
93 <br/> # file SimpleCalcParser.yy...</div>
94 <div class="OptionsBox">
95 $InputText = &quot;3 + 4 +6\nx=3\ny=5\nx+y\nx+z\n&quot;;</div>
96 <div class="OptionsBox">
97 $YYLexer = new Parsers::YYLexer($InputText,@LexerTokensSpec);</div>
98 <div class="OptionsBox">
99 $YYLex = $YYLexer-&gt;GetYYLex();</div>
100 <div class="OptionsBox">
101 $YYTabFile = &quot;Parsers/SimpleCalcParser.tab.ph&quot;;
102 <br/> $YYLexer-&gt;SetupYYTabFile($YYTabFile);</div>
103 <div class="OptionsBox">
104 $Debug = 0;
105 <br/> $SimpleCalcParser = new Parsers::SimpleCalcParser($YYLex,
106 \&amp;Parsers::SimpleCalcParser::yyerror, $Debug);</div>
107 <div class="OptionsBox">
108 $Value = $SimpleCalcParser-&gt;yyparse();
109 <br/> print &quot;Value = &quot; . (defined($Value) ? &quot;$Value&quot; : &quot;Undefined&quot;) . &quot;\n&quot;;</div>
110 </dd>
111 <dt><strong><a name="getyylex" class="item"><strong>GetYYLex</strong></a></strong></dt>
112 <dd>
113 <div class="OptionsBox">
114 $YYLex = $YYLexer-&gt;GetYYLex();</div>
115 <p>Returns a curried verson of YYLexer as <strong>YYLex</strong>: yyparse in parser generated by
116 byacc expects it to call without passing any argument for the <em>YYLexer</em> object.</p>
117 </dd>
118 <dt><strong><a name="next" class="item"><strong>Next</strong></a></strong></dt>
119 <dd>
120 <div class="OptionsBox">
121 ($TokenNumber, $TokenText) = $YYLexer-&gt;Next();</div>
122 <p>Returns next available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from
123 input stream by removing it from the input stream. Token number and text of
124 zero corresponds to end of input (EOI).</p>
125 </dd>
126 <dt><strong><a name="peek" class="item"><strong>Peek</strong></a></strong></dt>
127 <dd>
128 <div class="OptionsBox">
129 ($TokenNumber, $TokenText) = $YYLexer-&gt;Peek();</div>
130 <p>Returns next available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from
131 input stream by simply looking ahead and without removing it from the input stream.
132 Token number and text of zero corresponds to end of input (EOI).</p>
133 </dd>
134 <dt><strong><a name="setupyytabfile" class="item"><strong>SetupYYTabFile</strong></a></strong></dt>
135 <dd>
136 <div class="OptionsBox">
137 $YYLexer = $YYLexer-&gt;SetupYYTabFile($YYTabFile);</div>
138 <p>Processes token labels to integers data map in specified <em>YYTabFile</em> and returns
139 <em>YYLexer</em>.</p>
140 <p>Notes:</p>
141 <div class="OptionsBox">
142 . YYTabFile must be a complete path or available through @INC path in the
143 same directory where this package is located.
144 <br/> . Name of YYTabFile might start with any valid sub directory name in @INC
145 For example, &quot;Parsers/&lt;YYTablFile&gt;&quot; implies the tab file in parsers sub
146 directory under MayaChemTools lib directory as it would be already in @INC
147 path.
148 <br/> . YYTabFile must be explicitly set by the caller. The default YYTabFile name,
149 y.tab.ph, generated by byacc is not used implicitly to avoid confusion
150 among multiple distinct instances of YYLexer.
151 <br/> . YYTabFile is generated by byacc during its usage with -d options and
152 contains mapping of token codes to token names/labels. YYLexer used this
153 file to map token labels to token codes before returning token code and
154 value pair back to yyparse function used by byacc.
155 <br/> . User defined token numbers start from 257
156 <br/> . Token number for any user defined token EOI is mapped to its value before
157 default token number of 0 for EOI.</div>
158 <div class="OptionsBox">
159 The format of YYTabFile generated by byacc during generation of parser code in
160 <br/> Perl code is:</div>
161 <div class="OptionsBox">
162 ... ...
163 <br/> $NUMBER=257;
164 <br/> $ADDOP=258;
165 <br/> $SUBOP=259;
166 <br/> ... ..</div>
167 </dd>
168 <dt><strong><a name="yylex" class="item"><strong>YYLex</strong></a></strong></dt>
169 <dd>
170 <div class="OptionsBox">
171 ($TokenNumber, $TokenText) = $YYLexer-&gt;YYLex();
172 <br/> ($TokenNumber, $TokenText) = $YYLexer-&gt;YYLex($Mode);</div>
173 <p>Returns available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from
174 input stream by either removing it from the input stream or by simply looking
175 ahead and without removing it from the input stream. Token number and text of
176 zero corresponds to end of input (EOI).</p>
177 <p>Possible <em>Mode</em> values: <em>Peek, Next</em>. Default: <em>Next</em>.</p>
178 <p><em>YYLex</em> is designed to be used with <strong>yyparse</strong> code generated by running
179 <strong>byacc</strong> on a parsers defined using parser definition <strong>ParserName.yy</strong> file.</p>
180 <p>Notes:</p>
181 <div class="OptionsBox">
182 . Token label and value pairs returned by Lexer from by base class, which
183 can't be mapped to token labels specified in YYTabFile are ignored.
184 <br/> . Token text of length 1 returned by Lexer from base class without a
185 corresponding explicit token label, which can't be mapped to a token
186 number using Perl ord function, is ignored.</div>
187 </dd>
188 <dt><strong><a name="stringifyyylexer" class="item"><strong>StringifyYYLexer</strong></a></strong></dt>
189 <dd>
190 <div class="OptionsBox">
191 $YYLexerString = $YYLexer-&gt;StringifyYYLexer();</div>
192 <p>Returns a string containing information about <em>YYLexer</em> object.</p>
193 </dd>
194 </dl>
195 <p>
196 </p>
197 <h2>AUTHOR</h2>
198 <p><a href="mailto:msud@san.rr.com">Manish Sud</a></p>
199 <p>
200 </p>
201 <h2>SEE ALSO</h2>
202 <p><a href="./Lexer.html">Lexer.pm</a>,&nbsp<a href="./SimpleCalcYYLexer.html">SimpleCalcYYLexer.pm</a>,&nbsp<a href="./SimpleCalcParser.html">SimpleCalcParser.yy</a>
203 </p>
204 <p>
205 </p>
206 <h2>COPYRIGHT</h2>
207 <p>Copyright (C) 2015 Manish Sud. All rights reserved.</p>
208 <p>This file is part of MayaChemTools.</p>
209 <p>MayaChemTools is free software; you can redistribute it and/or modify it under
210 the terms of the GNU Lesser General Public License as published by the Free
211 Software Foundation; either version 3 of the License, or (at your option)
212 any later version.</p>
213 <p>&nbsp</p><p>&nbsp</p><div class="DocNav">
214 <table width="100%" border=0 cellpadding=0 cellspacing=2>
215 <tr align="left" valign="top"><td width="33%" align="left"><a href="./SimpleCalcYYLexer.html" title="SimpleCalcYYLexer.html">Previous</a>&nbsp;&nbsp;<a href="./index.html" title="Table of Contents">TOC</a>&nbsp;&nbsp;<a href="./PDBFileUtil.html" title="PDBFileUtil.html">Next</a></td><td width="34%" align="middle"><strong>March 29, 2015</strong></td><td width="33%" align="right"><strong>Parsers::YYLexer.pm</strong></td></tr>
216 </table>
217 </div>
218 <br />
219 <center>
220 <img src="../../images/h2o2.png">
221 </center>
222 </body>
223 </html>