view mayachemtools/docs/modules/man3/YYLexer.3 @ 9:ab29fa5c8c1f draft default tip

Uploaded

.\" Automatically generated by Pod::Man 2.25 (Pod::Simple 3.22)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is turned on, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.ie \nF \{\
.    de IX
.    tm Index:\\$1\t\\n%\t"\\$2"
..
.    nr % 0
.    rr F
.\}
.el \{\
.    de IX
..
.\}
.\"
.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
.\" Fear.  Run.  Save yourself.  No user-serviceable parts.
.    \" fudge factors for nroff and troff
.if n \{\
.    ds #H 0
.    ds #V .8m
.    ds #F .3m
.    ds #[ \f1
.    ds #] \fP
.\}
.if t \{\
.    ds #H ((1u-(\\\\n(.fu%2u))*.13m)
.    ds #V .6m
.    ds #F 0
.    ds #[ \&
.    ds #] \&
.\}
.    \" simple accents for nroff and troff
.if n \{\
.    ds ' \&
.    ds ` \&
.    ds ^ \&
.    ds , \&
.    ds ~ ~
.    ds /
.\}
.if t \{\
.    ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
.    ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
.    ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
.    ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
.    ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
.    ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
.\}
.    \" troff and (daisy-wheel) nroff accents
.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
.ds ae a\h'-(\w'a'u*4/10)'e
.ds Ae A\h'-(\w'A'u*4/10)'E
.    \" corrections for vroff
.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
.    \" for low resolution devices (crt and lpr)
.if \n(.H>23 .if \n(.V>19 \
\{\
.    ds : e
.    ds 8 ss
.    ds o a
.    ds d- d\h'-1'\(ga
.    ds D- D\h'-1'\(hy
.    ds th \o'bp'
.    ds Th \o'LP'
.    ds ae ae
.    ds Ae AE
.\}
.rm #[ #] #H #V #F C
.\" ========================================================================
.\"
.IX Title "YYLEXER 1"
.TH YYLEXER 1 "2015-03-29" "perl v5.14.2" "MayaChemTools"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
Parsers::YYLexer
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
use Parseres::YYLexer;
.PP
use Parsers::YYLexer qw(:all);
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
\&\fBYYLexer\fR class provides the following methods:
.PP
new, GetYYLex, Next, Peek, SetupYYTabFile, StringifyYYLexer, YYLex
.PP
\&\fBParsers::YYLexer\fR class is derived from \fBParsers::Lexer\fR base class, which provides all
the underlying lexer functionality. \fBYYLexer\fR class is designed to be used with
\&\fByyparse\fR code generated by running \fBbyacc\fR on a parsers defined using
parser definition \fBParserName.yy\fR file.
.PP
\&\fIYYTabFile\fR containing mapping of token labels to integers must be explicitly
specified by the caller. This file is processed during new method invocation and
mapping of token labels to integers is loaded in a hash to be used later by \fBYYLex\fR
method to return token number and text pairs to the parser.
.SS "\s-1METHODS\s0"
.IX Subsection "METHODS"
.IP "\fBnew\fR" 4
.IX Item "new"
.Vb 1
\&    $YYLexer = new Parsers::YYLexer($Input,  @YYLexerTokensSpec);
.Ve
.Sp
Using specified \fIInput\fR and \fIYYLexerTokensSpec\fR, \fBnew\fR method generates a new
\&\fBYYLexer\fR and returns a reference to newly created \fBYYLexer\fR object.
.Sp
Examples:
.Sp
.Vb 10
\&    # Tokens specifications supplied by the caller. It\*(Aqs an array containing references
\&    # to arrays with each containing TokenLabel and TokenMatchRegex pair along with
\&    # an option reference to code to be executed after a matched.
\&    #
\&    @LexerTokensSpec = (
\&        [ \*(AqLETTER\*(Aq, qr/[a\-zA\-Z]/ ],
\&        [ \*(AqNUMBER\*(Aq, qr/\ed+/ ],
\&        [ \*(AqSPACE\*(Aq, qr/[ ]*/,
\&            sub { my($This, $TokenLabel, $MatchedText) = @_; return \*(Aq\*(Aq; }
\&        ],
\&        [ \*(AqNEWLINE\*(Aq, qr/(?:\er\en|\er|\en)/,
\&            sub { my($This, $TokenLabel, $MatchedText) = @_;  return "\en"; }
\&        ],
\&        [ \*(AqCHAR\*(Aq, qr/./ ]
\&    );
\&
\&    # Input string...
\&    $InputText = \*(Aqy = 3 + 4\*(Aq;
\&
\&    $YLexer = new Parsers::YYLexer($InputText,  @LexerTokensSpec);
\&
\&    # Setup default token table file...
\&    $YYTabFile = "Parsers/SimpleCalcParser.tab.ph";
\&    $This\->SetupYYTabFile($YYTabFile);
\&
\&    # Process input stream...
\&    ($TokenNumber, $TokenText) = $YYLexer\->Lex();
\&    print "TokenNumber: $TokenNumber; TokenText: $TokenText\en";
\&
\&    # Input file...
\&    $InputFile = "Input.txt";
\&    open INPUTFILE, "$InputFile" or die "Couldn\*(Aqt open $InputFile: $!\en";
\&    $Lexer = new Parsers::YYLexer(\e*INPUTFILE, @LexerTokensSpec);
\&
\&    # Input file iterator...
\&    $InputFile = "TestSimpleCalcParser.txt";
\&    open INPUTFILE, "$InputFile" or die "Couldn\*(Aqt open $InputFile: $!\en";
\&    $InputIterator = sub { return <INPUTFILE>; };
\&    $Lexer = new Parsers::YYLexer($InputIterator, @LexerTokensSpec);
\&
\&    # Usage with code generated by byacc from a parser definition
\&    # file SimpleCalcParser.yy...
\&
\&    $InputText = "3 + 4 +6\enx=3\eny=5\enx+y\enx+z\en";
\&
\&    $YYLexer = new Parsers::YYLexer($InputText,@LexerTokensSpec);
\&
\&    $YYLex = $YYLexer\->GetYYLex();
\&
\&    $YYTabFile = "Parsers/SimpleCalcParser.tab.ph";
\&    $YYLexer\->SetupYYTabFile($YYTabFile);
\&
\&    $Debug = 0;
\&    $SimpleCalcParser = new Parsers::SimpleCalcParser($YYLex,
\&                            \e&Parsers::SimpleCalcParser::yyerror, $Debug);
\&
\&    $Value = $SimpleCalcParser\->yyparse();
\&    print "Value = " . (defined($Value) ? "$Value" : "Undefined") . "\en";
.Ve
.IP "\fBGetYYLex\fR" 4
.IX Item "GetYYLex"
.Vb 1
\&    $YYLex = $YYLexer\->GetYYLex();
.Ve
.Sp
Returns a curried verson of YYLexer as \fBYYLex\fR: yyparse in parser generated by
byacc expects it to call without passing any argument for the \fIYYLexer\fR object.
.IP "\fBNext\fR" 4
.IX Item "Next"
.Vb 1
\&    ($TokenNumber, $TokenText) = $YYLexer\->Next();
.Ve
.Sp
Returns next available \fBTokenNumber\fR and any matched \fBTokenText\fR from
input stream by removing it from the input stream. Token number and text of
zero corresponds to end of input (\s-1EOI\s0).
.IP "\fBPeek\fR" 4
.IX Item "Peek"
.Vb 1
\&    ($TokenNumber, $TokenText) = $YYLexer\->Peek();
.Ve
.Sp
Returns next available \fBTokenNumber\fR and any matched \fBTokenText\fR from
input stream by simply looking ahead and without removing it from the input stream.
Token number and text of zero corresponds to end of input (\s-1EOI\s0).
.IP "\fBSetupYYTabFile\fR" 4
.IX Item "SetupYYTabFile"
.Vb 1
\&    $YYLexer = $YYLexer\->SetupYYTabFile($YYTabFile);
.Ve
.Sp
Processes token labels to integers data map in specified \fIYYTabFile\fR and returns
\&\fIYYLexer\fR.
.Sp
Notes:
.Sp
.Vb 10
\&    . YYTabFile must be a complete path or available through @INC path in the
\&      same directory where this package is located.
\&    . Name of YYTabFile might start with any valid sub directory name in @INC
\&      For example, "Parsers/<YYTablFile>" implies the tab file in parsers sub
\&      directory under MayaChemTools lib directory as it would be already in @INC
\&      path.
\&    . YYTabFile must be explicitly set by the caller. The default YYTabFile name,
\&      y.tab.ph, generated by byacc is not used implicitly to avoid confusion
\&      among multiple distinct instances of YYLexer.
\&    . YYTabFile is generated by byacc during its usage with \-d options and
\&      contains mapping of token codes to token names/labels. YYLexer used this
\&      file to map token labels to token codes before returning token code and
\&      value pair back to yyparse function used by byacc.
\&    . User defined token numbers start from 257
\&    . Token number for any user defined token EOI is mapped to its value before
\&      default token number of 0 for EOI.
\&
\&    The format of YYTabFile generated by byacc during generation of parser code in
\&    Perl code is:
\&
\&    ... ...
\&    $NUMBER=257;
\&    $ADDOP=258;
\&    $SUBOP=259;
\&    ... ..
.Ve
.IP "\fBYYLex\fR" 4
.IX Item "YYLex"
.Vb 2
\&    ($TokenNumber, $TokenText) = $YYLexer\->YYLex();
\&    ($TokenNumber, $TokenText) = $YYLexer\->YYLex($Mode);
.Ve
.Sp
Returns available \fBTokenNumber\fR and any matched \fBTokenText\fR from
input stream by either removing it from the input stream or by simply looking
ahead and without removing it from the input stream. Token number and text of
zero corresponds to end of input (\s-1EOI\s0).
.Sp
Possible \fIMode\fR values: \fIPeek, Next\fR. Default: \fINext\fR.
.Sp
\&\fIYYLex\fR is designed to be used with \fByyparse\fR code generated by running
\&\fBbyacc\fR on a parsers defined using parser definition \fBParserName.yy\fR file.
.Sp
Notes:
.Sp
.Vb 5
\&    . Token label and value pairs returned by Lexer from by base class, which
\&       can\*(Aqt be mapped to token labels specified in YYTabFile are ignored.
\&    . Token text of length 1 returned by Lexer from base class without a
\&       corresponding explicit token label, which can\*(Aqt be mapped to a token
\&       number using Perl ord function, is ignored.
.Ve
.IP "\fBStringifyYYLexer\fR" 4
.IX Item "StringifyYYLexer"
.Vb 1
\&    $YYLexerString = $YYLexer\->StringifyYYLexer();
.Ve
.Sp
Returns a string containing information about \fIYYLexer\fR object.
.SH "AUTHOR"
.IX Header "AUTHOR"
Manish Sud <[e-mail]>
.SH "SEE ALSO"
.IX Header "SEE ALSO"
Lexer.pm, SimpleCalcYYLexer.pm, SimpleCalcParser.yy
.SH "COPYRIGHT"
.IX Header "COPYRIGHT"
Copyright (C) 2015 Manish Sud. All rights reserved.
.PP
This file is part of MayaChemTools.
.PP
MayaChemTools is free software; you can redistribute it and/or modify it under
the terms of the \s-1GNU\s0 Lesser General Public License as published by the Free
Software Foundation; either version 3 of the License, or (at your option)
any later version.