Lexical Analysis - GitHub Pages
嚜燉exical Analysis
? Recognize tokens and ignore white spaces,
comments
Generates token stream
? Error reporting
? Model using regular expressions
? Recognize using Finite State Automata1
Lexical Analysis
? Sentences consist of string of tokens (a
syntactic category)
For example, number, identifier, keyword,
string
? Sequences of characters in a token is a
lexeme
for example, 100.01, counter, const,
※How are you?§
? Rule of description is a pattern
for example, letter ( letter | digit )*
? Task: Identify Tokens and corresponding
Lexemes
2
Lexical Analysis
? Examples
? Construct constants: for example, convert a
number to token num and pass the value as its
attribute,
每 31 becomes
? Recognize keyword and identifiers
每 counter = counter + increment
becomes id = id + id
每 check that id here is not a keyword
? Discard whatever does not contribute to
parsing
每 white spaces (blanks, tabs, newlines) and
comments
3
Interface to other phases
Read
characters
Input
Push back
Extra
characters
Token
Lexical
Analyzer
Syntax
Analyzer
Ask for
token
? Why do we need Push back?
? Required due to look-ahead
for example, to recognize >= and >
? Typically implemented through a buffer
每 Keep input in a buffer
每 Move pointers over the input
4
Approaches to implementation
? Use assembly language
Most efficient but most difficult to implement
? Use high level languages like C
Efficient but difficult to implement
? Use tools like lex, flex
Easy to implement but not as efficient as the first
two cases
5
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related download
- sympy rxjs ggplot2 python data persistence caffe2
- convert xy type safe interchange of c and python
- python language syntax cheat sheet
- introduction to python
- lecture 2 python data type and variables data data type
- chapter 3 input and type conversion
- a guide to f string formatting in python
- logix 5000 controllers ascii strings 1756 pm013g en p
- lexical analysis github pages
- python day 2 iwks