Index (Frames) | Index (No Frames) | Package | Package Tree | Tree
java.io

Class StreamTokenizer

java.lang.Object
|
+--java.io.StreamTokenizer


public class StreamTokenizer

extends Object

This class parses streams of characters into tokens. There are a million-zillion flags that can be set to control the parsing, as described under the various method headings.

Author:

Field Summary

static intTT_EOF

A constant indicating that the end of the stream has been read.
static intTT_EOL

A constant indicating that the end of the line has been read.
static intTT_NUMBER

A constant indicating that a number token has been read.
static intTT_WORD

A constant indicating that a word token has been read.
doublenval

The numeric value associated with number tokens.
java.lang.Stringsval

The String associated with word and string tokens.
intttype

Contains the type of the token read resulting from a call to nextToken The rules are as follows:
  • For a token consisting of a single ordinary character, this is the value of that character.

Constructor Summary

StreamTokenizer(java.io.InputStream is)

This method reads bytes from an InputStream and tokenizes them.
StreamTokenizer(java.io.Reader r)

This method initializes a new StreamTokenizer to read characters from a Reader and parse them.

Method Summary

voidcommentChar(int ch)

This method sets the comment attribute on the specified character.
voideolIsSignificant(boolean flag)

This method sets a flag that indicates whether or not the end of line sequence terminates and is a token.
intlineno()

This method returns the current line number.
voidlowerCaseMode(boolean flag)

This method sets a flag that indicates whether or not alphabetic tokens that are returned should be converted to lower case.
intnextToken()

This method reads the next token from the stream.
voidordinaryChar(int ch)

This method makes the specified character an ordinary character.
voidordinaryChars(int low, int hi)

This method makes all the characters in the specified range, range terminators included, ordinary.
voidparseNumbers()

This method sets the numeric attribute on the characters '0' - '9' and the characters '.' and '-'.
voidpushBack()

Puts the current token back into the StreamTokenizer so nextToken will return the same value on the next call.
voidquoteChar(int ch)

This method sets the quote attribute on the specified character.
voidresetSyntax()

This method removes all attributes (whitespace, alphabetic, numeric, quote, and comment) from all characters.
voidslashSlashComments(boolean flag)

This method sets a flag that indicates whether or not "C++" language style comments ("//" comments through EOL ) are handled by the parser.
voidslashStarComments(boolean flag)

This method sets a flag that indicates whether or not "C" language style comments (with nesting not allowed) are handled by the parser.
java.lang.StringtoString()

This method returns the current token value as a String in the form "Token[x], line n", where 'n' is the current line numbers and 'x' is determined as follows.
voidwhitespaceChars(int low, int hi)

This method sets the whitespace attribute for all characters in the specified range, range terminators included.
voidwordChars(int low, int hi)

This method sets the alphabetic attribute for all characters in the specified range, range terminators included.

Field Details

TT_EOF

public static final int TT_EOF

A constant indicating that the end of the stream has been read.


TT_EOL

public static final int TT_EOL

A constant indicating that the end of the line has been read.


TT_NUMBER

public static final int TT_NUMBER

A constant indicating that a number token has been read.


TT_WORD

public static final int TT_WORD

A constant indicating that a word token has been read.


nval

public double nval

The numeric value associated with number tokens.


sval

public String sval

The String associated with word and string tokens.


ttype

public int ttype

Contains the type of the token read resulting from a call to nextToken The rules are as follows:


Constructor Details

StreamTokenizer

public StreamTokenizer(java.io.InputStream is)

This method reads bytes from an InputStream and tokenizes them. For details on how this method operates by default, see StreamTokenizer(Reader).

Parameters:


StreamTokenizer

public StreamTokenizer(java.io.Reader r)

This method initializes a new StreamTokenizer to read characters from a Reader and parse them. The char values have their hight bits masked so that the value is treated a character in the range of 0x0000 to 0x00FF.

This constructor sets up the parsing table to parse the stream in the following manner:

Parameters:


Method Details

commentChar

public void commentChar(int ch)

This method sets the comment attribute on the specified character.

Parameters:


eolIsSignificant

public void eolIsSignificant(boolean flag)

This method sets a flag that indicates whether or not the end of line sequence terminates and is a token. The defaults to false

Parameters:


lineno

public int lineno()

This method returns the current line number. Note that if the pushBack() method is called, it has no effect on the line number returned by this method.

Returns:


lowerCaseMode

public void lowerCaseMode(boolean flag)

This method sets a flag that indicates whether or not alphabetic tokens that are returned should be converted to lower case.

Parameters:


nextToken

public int nextToken()

This method reads the next token from the stream. It sets the ttype variable to the appropriate token type and returns it. It also can set sval or nval as described below. The parsing strategy is as follows:

Returns:

Throws:


ordinaryChar

public void ordinaryChar(int ch)

This method makes the specified character an ordinary character. This means that none of the attributes (whitespace, alphabetic, numeric, quote, or comment) will be set on this character. This character will parse as its own token.

Parameters:


ordinaryChars

public void ordinaryChars(int low, int hi)

This method makes all the characters in the specified range, range terminators included, ordinary. This means the none of the attributes (whitespace, alphabetic, numeric, quote, or comment) will be set on any of the characters in the range. This makes each character in this range parse as its own token.

Parameters:


parseNumbers

public void parseNumbers()

This method sets the numeric attribute on the characters '0' - '9' and the characters '.' and '-'.


pushBack

public void pushBack()

Puts the current token back into the StreamTokenizer so nextToken will return the same value on the next call. May cause the lineno method to return an incorrect value if lineno is called before the next call to nextToken.


quoteChar

public void quoteChar(int ch)

This method sets the quote attribute on the specified character.

Parameters:


resetSyntax

public void resetSyntax()

This method removes all attributes (whitespace, alphabetic, numeric, quote, and comment) from all characters. It is equivalent to calling ordinaryChars(0x00, 0xFF).

See Also:


slashSlashComments

public void slashSlashComments(boolean flag)

This method sets a flag that indicates whether or not "C++" language style comments ("//" comments through EOL ) are handled by the parser. If this is true commented out sequences are skipped and ignored by the parser. This defaults to false.

Parameters:


slashStarComments

public void slashStarComments(boolean flag)

This method sets a flag that indicates whether or not "C" language style comments (with nesting not allowed) are handled by the parser. If this is true commented out sequences are skipped and ignored by the parser. This defaults to false.

Parameters:


toString

public String toString()

This method returns the current token value as a String in the form "Token[x], line n", where 'n' is the current line numbers and 'x' is determined as follows.


whitespaceChars

public void whitespaceChars(int low, int hi)

This method sets the whitespace attribute for all characters in the specified range, range terminators included.

Parameters:


wordChars

public void wordChars(int low, int hi)

This method sets the alphabetic attribute for all characters in the specified range, range terminators included.

Parameters: