Mercurial > repos > deepakjadmin > mayatool3_test3
diff mayachemtools/docs/modules/html/YYLexer.html @ 0:73ae111cf86f draft
Uploaded
| author | deepakjadmin |
|---|---|
| date | Wed, 20 Jan 2016 11:55:01 -0500 |
| parents | |
| children |
line wrap: on
line diff
--- /dev/null Thu Jan 01 00:00:00 1970 +0000 +++ b/mayachemtools/docs/modules/html/YYLexer.html Wed Jan 20 11:55:01 2016 -0500 @@ -0,0 +1,223 @@ +<html> +<head> +<title>MayaChemTools:Documentation:Parsers::YYLexer.pm</title> +<meta http-equiv="content-type" content="text/html;charset=utf-8"> +<link rel="stylesheet" type="text/css" href="../../css/MayaChemTools.css"> +</head> +<body leftmargin="20" rightmargin="20" topmargin="10" bottommargin="10"> +<br/> +<center> +<a href="http://www.mayachemtools.org" title="MayaChemTools Home"><img src="../../images/MayaChemToolsLogo.gif" border="0" alt="MayaChemTools"></a> +</center> +<br/> +<div class="DocNav"> +<table width="100%" border=0 cellpadding=0 cellspacing=2> +<tr align="left" valign="top"><td width="33%" align="left"><a href="./SimpleCalcYYLexer.html" title="SimpleCalcYYLexer.html">Previous</a> <a href="./index.html" title="Table of Contents">TOC</a> <a href="./PDBFileUtil.html" title="PDBFileUtil.html">Next</a></td><td width="34%" align="middle"><strong>Parsers::YYLexer.pm</strong></td><td width="33%" align="right"><a href="././code/YYLexer.html" title="View source code">Code</a> | <a href="./../pdf/YYLexer.pdf" title="PDF US Letter Size">PDF</a> | <a href="./../pdfgreen/YYLexer.pdf" title="PDF US Letter Size with narrow margins: www.changethemargins.com">PDFGreen</a> | <a href="./../pdfa4/YYLexer.pdf" title="PDF A4 Size">PDFA4</a> | <a href="./../pdfa4green/YYLexer.pdf" title="PDF A4 Size with narrow margins: www.changethemargins.com">PDFA4Green</a></td></tr> +</table> +</div> +<p> +</p> +<h2>NAME</h2> +<p>Parsers::YYLexer</p> +<p> +</p> +<h2>SYNOPSIS</h2> +<p>use Parseres::YYLexer;</p> +<p>use Parsers::YYLexer qw(:all);</p> +<p> +</p> +<h2>DESCRIPTION</h2> +<p><strong>YYLexer</strong> class provides the following methods:</p> +<p> <a href="#new">new</a>, <a href="#getyylex">GetYYLex</a>, <a href="#next">Next</a>, <a href="#peek">Peek</a>, <a href="#setupyytabfile">SetupYYTabFile</a>, <a href="#stringifyyylexer">StringifyYYLexer</a>, <a href="#yylex">YYLex</a> +</p><p><strong>Parsers::YYLexer</strong> class is derived from <strong>Parsers::Lexer</strong> base class, which provides all +the underlying lexer functionality. <strong>YYLexer</strong> class is designed to be used with +<strong>yyparse</strong> code generated by running <strong>byacc</strong> on a parsers defined using +parser definition <strong>ParserName.yy</strong> file.</p> +<p><em>YYTabFile</em> containing mapping of token labels to integers must be explicitly +specified by the caller. This file is processed during new method invocation and +mapping of token labels to integers is loaded in a hash to be used later by <strong>YYLex</strong> +method to return token number and text pairs to the parser.</p> +<p> +</p> +<h2>METHODS</h2> +<dl> +<dt><strong><a name="new" class="item"><strong>new</strong></a></strong></dt> +<dd> +<div class="OptionsBox"> + $YYLexer = new Parsers::YYLexer($Input, @YYLexerTokensSpec);</div> +<p>Using specified <em>Input</em> and <em>YYLexerTokensSpec</em>, <strong>new</strong> method generates a new +<strong>YYLexer</strong> and returns a reference to newly created <strong>YYLexer</strong> object.</p> +<p>Examples:</p> +<div class="OptionsBox"> + # Tokens specifications supplied by the caller. It's an array containing references +<br/> # to arrays with each containing TokenLabel and TokenMatchRegex pair along with +<br/> # an option reference to code to be executed after a matched. +<br/> # +<br/> @LexerTokensSpec = ( +<br/> [ 'LETTER', qr/[a-zA-Z]/ ], + [ 'NUMBER', qr/\d+/ ], + [ 'SPACE', qr/[ ]*/, +<br/> sub { my($This, $TokenLabel, $MatchedText) = @_; return ''; } +<br/> ], + [ 'NEWLINE', qr/(?:\r\n|\r|\n)/, +<br/> sub { my($This, $TokenLabel, $MatchedText) = @_; return "\n"; } +<br/> ], + [ 'CHAR', qr/./ ] +<br/> );</div> +<div class="OptionsBox"> + # Input string... +<br/> $InputText = 'y = 3 + 4';</div> +<div class="OptionsBox"> + $YLexer = new Parsers::YYLexer($InputText, @LexerTokensSpec);</div> +<div class="OptionsBox"> + # Setup default token table file... +<br/> $YYTabFile = "Parsers/SimpleCalcParser.tab.ph"; +<br/> $This->SetupYYTabFile($YYTabFile);</div> +<div class="OptionsBox"> + # Process input stream... +<br/> ($TokenNumber, $TokenText) = $YYLexer->Lex(); +<br/> print "TokenNumber: $TokenNumber; TokenText: $TokenText\n";</div> +<div class="OptionsBox"> + # Input file... +<br/> $InputFile = "Input.txt"; +<br/> open INPUTFILE, "$InputFile" or die "Couldn't open $InputFile: $!\n"; +<br/> $Lexer = new Parsers::YYLexer(\*INPUTFILE, @LexerTokensSpec);</div> +<div class="OptionsBox"> + # Input file iterator... +<br/> $InputFile = "TestSimpleCalcParser.txt"; +<br/> open INPUTFILE, "$InputFile" or die "Couldn't open $InputFile: $!\n"; +<br/> $InputIterator = sub { return <INPUTFILE>; }; +<br/> $Lexer = new Parsers::YYLexer($InputIterator, @LexerTokensSpec);</div> +<div class="OptionsBox"> + # Usage with code generated by byacc from a parser definition +<br/> # file SimpleCalcParser.yy...</div> +<div class="OptionsBox"> + $InputText = "3 + 4 +6\nx=3\ny=5\nx+y\nx+z\n";</div> +<div class="OptionsBox"> + $YYLexer = new Parsers::YYLexer($InputText,@LexerTokensSpec);</div> +<div class="OptionsBox"> + $YYLex = $YYLexer->GetYYLex();</div> +<div class="OptionsBox"> + $YYTabFile = "Parsers/SimpleCalcParser.tab.ph"; +<br/> $YYLexer->SetupYYTabFile($YYTabFile);</div> +<div class="OptionsBox"> + $Debug = 0; +<br/> $SimpleCalcParser = new Parsers::SimpleCalcParser($YYLex, + \&Parsers::SimpleCalcParser::yyerror, $Debug);</div> +<div class="OptionsBox"> + $Value = $SimpleCalcParser->yyparse(); +<br/> print "Value = " . (defined($Value) ? "$Value" : "Undefined") . "\n";</div> +</dd> +<dt><strong><a name="getyylex" class="item"><strong>GetYYLex</strong></a></strong></dt> +<dd> +<div class="OptionsBox"> + $YYLex = $YYLexer->GetYYLex();</div> +<p>Returns a curried verson of YYLexer as <strong>YYLex</strong>: yyparse in parser generated by +byacc expects it to call without passing any argument for the <em>YYLexer</em> object.</p> +</dd> +<dt><strong><a name="next" class="item"><strong>Next</strong></a></strong></dt> +<dd> +<div class="OptionsBox"> + ($TokenNumber, $TokenText) = $YYLexer->Next();</div> +<p>Returns next available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from +input stream by removing it from the input stream. Token number and text of +zero corresponds to end of input (EOI).</p> +</dd> +<dt><strong><a name="peek" class="item"><strong>Peek</strong></a></strong></dt> +<dd> +<div class="OptionsBox"> + ($TokenNumber, $TokenText) = $YYLexer->Peek();</div> +<p>Returns next available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from +input stream by simply looking ahead and without removing it from the input stream. +Token number and text of zero corresponds to end of input (EOI).</p> +</dd> +<dt><strong><a name="setupyytabfile" class="item"><strong>SetupYYTabFile</strong></a></strong></dt> +<dd> +<div class="OptionsBox"> + $YYLexer = $YYLexer->SetupYYTabFile($YYTabFile);</div> +<p>Processes token labels to integers data map in specified <em>YYTabFile</em> and returns +<em>YYLexer</em>.</p> +<p>Notes:</p> +<div class="OptionsBox"> + . YYTabFile must be a complete path or available through @INC path in the + same directory where this package is located. +<br/> . Name of YYTabFile might start with any valid sub directory name in @INC + For example, "Parsers/<YYTablFile>" implies the tab file in parsers sub + directory under MayaChemTools lib directory as it would be already in @INC + path. +<br/> . YYTabFile must be explicitly set by the caller. The default YYTabFile name, + y.tab.ph, generated by byacc is not used implicitly to avoid confusion + among multiple distinct instances of YYLexer. +<br/> . YYTabFile is generated by byacc during its usage with -d options and + contains mapping of token codes to token names/labels. YYLexer used this + file to map token labels to token codes before returning token code and + value pair back to yyparse function used by byacc. +<br/> . User defined token numbers start from 257 +<br/> . Token number for any user defined token EOI is mapped to its value before + default token number of 0 for EOI.</div> +<div class="OptionsBox"> + The format of YYTabFile generated by byacc during generation of parser code in +<br/> Perl code is:</div> +<div class="OptionsBox"> + ... ... +<br/> $NUMBER=257; +<br/> $ADDOP=258; +<br/> $SUBOP=259; +<br/> ... ..</div> +</dd> +<dt><strong><a name="yylex" class="item"><strong>YYLex</strong></a></strong></dt> +<dd> +<div class="OptionsBox"> + ($TokenNumber, $TokenText) = $YYLexer->YYLex(); +<br/> ($TokenNumber, $TokenText) = $YYLexer->YYLex($Mode);</div> +<p>Returns available <strong>TokenNumber</strong> and any matched <strong>TokenText</strong> from +input stream by either removing it from the input stream or by simply looking +ahead and without removing it from the input stream. Token number and text of +zero corresponds to end of input (EOI).</p> +<p>Possible <em>Mode</em> values: <em>Peek, Next</em>. Default: <em>Next</em>.</p> +<p><em>YYLex</em> is designed to be used with <strong>yyparse</strong> code generated by running +<strong>byacc</strong> on a parsers defined using parser definition <strong>ParserName.yy</strong> file.</p> +<p>Notes:</p> +<div class="OptionsBox"> + . Token label and value pairs returned by Lexer from by base class, which + can't be mapped to token labels specified in YYTabFile are ignored. +<br/> . Token text of length 1 returned by Lexer from base class without a + corresponding explicit token label, which can't be mapped to a token + number using Perl ord function, is ignored.</div> +</dd> +<dt><strong><a name="stringifyyylexer" class="item"><strong>StringifyYYLexer</strong></a></strong></dt> +<dd> +<div class="OptionsBox"> + $YYLexerString = $YYLexer->StringifyYYLexer();</div> +<p>Returns a string containing information about <em>YYLexer</em> object.</p> +</dd> +</dl> +<p> +</p> +<h2>AUTHOR</h2> +<p><a href="mailto:msud@san.rr.com">Manish Sud</a></p> +<p> +</p> +<h2>SEE ALSO</h2> +<p><a href="./Lexer.html">Lexer.pm</a>, <a href="./SimpleCalcYYLexer.html">SimpleCalcYYLexer.pm</a>, <a href="./SimpleCalcParser.html">SimpleCalcParser.yy</a> +</p> +<p> +</p> +<h2>COPYRIGHT</h2> +<p>Copyright (C) 2015 Manish Sud. All rights reserved.</p> +<p>This file is part of MayaChemTools.</p> +<p>MayaChemTools is free software; you can redistribute it and/or modify it under +the terms of the GNU Lesser General Public License as published by the Free +Software Foundation; either version 3 of the License, or (at your option) +any later version.</p> +<p> </p><p> </p><div class="DocNav"> +<table width="100%" border=0 cellpadding=0 cellspacing=2> +<tr align="left" valign="top"><td width="33%" align="left"><a href="./SimpleCalcYYLexer.html" title="SimpleCalcYYLexer.html">Previous</a> <a href="./index.html" title="Table of Contents">TOC</a> <a href="./PDBFileUtil.html" title="PDBFileUtil.html">Next</a></td><td width="34%" align="middle"><strong>March 29, 2015</strong></td><td width="33%" align="right"><strong>Parsers::YYLexer.pm</strong></td></tr> +</table> +</div> +<br /> +<center> +<img src="../../images/h2o2.png"> +</center> +</body> +</html>