Tokenizer in Linked List - California State University ...
Tokenizer in Linked List
The following is a program of one student who implemented successful the tokenizer. However, the class is not clean to be used by other classes. Then you will see a major modified version. Many unnecessary variables and structures are removed. Here are some general commonest:
1. The original program has a bug on the link tokenPre.
2. It does not handle the hidden token multiplication in the case of 125x or x125.
3. Attributes and constructors should be built to alleviate the burden on the main program. The main program suppose to be a testing or guideline for the class.
4. Many sections of codes are repeating. They should be moved to a method as a single copy so that it can be centralized for modification.
5. The constants for the token type 0-9 should be centralized as well. Instead using the values, implementers should used the constant statement to code them such as
CONST MULT = 4;
CONST PLUS = 2;
Two appendices are given:
A. A modified version of Tokenizer
B. The stack to be implemented in linked list.
Appendix A.
//Modified
//11-23-99
// source inpostfix.java
// infix order to postfix order
import java.io.*;
import java.lang.*;
import java.util.*;
class TokenAndType
{
// a pair as attributes
String token;
int type;
//constructor
public TokenAndType()
{
token = null;
type = -1; // means no type
} // end of contructor
public TokenAndType(String tokenName,int tokenType)
{
token = tokenName;
type = tokenType;
} // end of contructor
public void printTokenPair()
{
System.out.print("token pair: " + token + " " + type);
}
} // end of class
class NodePtr
{
// Attributes of any member of NodePtr
TokenAndType tokenPair;
NodePtr next;
// Constructor to insert the node at the top of inputNodePtr
public NodePtr(TokenAndType tokenPair, NodePtr next)
{
this.tokenPair = tokenPair;
this.next = next;
}
public NodePtr()
{
tokenPair = new TokenAndType();
next = null;
}
public NodePtr( String tokenName,int tokenType)
{
tokenPair = new TokenAndType(tokenName, tokenType);
next = null;
}
}
class Tokenizer
{
// attributes
NodePtr top = null; // this is the tokenStream in Linked list
// method
public void GetTokens()
{
NodePtr temp = null;
NodePtr last = null; // points to the last node
String token=""; // from string of characters, cut into 1 token
// at a time
InputStreamReader reader = new InputStreamReader (System.in);
BufferedReader console = new BufferedReader (reader);
System.out.print("please enter data> ");
String StrExp = null;
try
{
StrExp = console.readLine();
}
catch (IOException e)
{
System.out.println("we call an" + e);
System.exit(1);
}
char firstChar=' ';
if (StrExp.length()!=0)firstChar = StrExp.charAt(0);
while(StrExp!=null)
{
switch (firstChar)
{
case '.': case '0': case '1': case '2': case '3':
case '4': case '5': case '6': case '7': case '8': case '9':
do {
token=token+firstChar;
if (StrExp.length()==1){StrExp = null;}
else
{
StrExp=StrExp.substring(1);
firstChar = StrExp.charAt(0);
}
}while( ((firstChar >='0' && firstChar ................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.
Related searches
- california state university system
- california state university second bachelor s
- california state university tuition
- california state university jobs
- california state university system schools
- california state university system wiki
- california state university application log in
- california state university campuses list
- california state university log in
- california state university application deadline
- california state university tuition fee
- california state university fees