com.inductiveautomation.snap.parse

Class Tokenizer

java.lang.Object

com.inductiveautomation.snap.parse.Tokenizer

Direct Known Subclasses:

JavaTokenizer

public class Tokenizer
extends java.lang.Object

A class to extract tokens from a char sequence.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	_charIndex
protected int	_lineIndex
protected int	_lineStart

Constructor Summary

Constructors

Constructor and Description
Tokenizer()

Method Summary

Modifier and Type	Method and Description
void	addPattern(java.lang.String aName, java.lang.String aPattern, boolean isLiteral) Adds a pattern.
void	addPatterns(ParseRule aRule) Adds patterns to this tokenizer for given rule.
char	charAt(int anIndex) CharSequence method.
protected Token	createToken(java.lang.String aName, java.lang.String aPattern, int aStart, int anEnd, Token aSpclTkn) Creates a new token.
char	getChar() Returns the current parse char.
int	getCharIndex() Returns the current parse char location.
int	getColumnIndex() Returns the current column index in the current line.
int	getColumnNumber() Returns the current column number.
java.lang.CharSequence	getInput() Returns the current tokenizer input.
java.lang.CharSequence	getInput(int aStart, int anEnd) Returns the input subsequence for the given range of characters in input.
int	getLineIndex() Returns the current line index.
int	getLineNumber() Returns the current line number.
int	getLineStart() Returns the current line start index.
Regex[]	getMatchers(char aChar) Returns list of matchers for a char.
Token	getNextSpecialToken() Processes and returns next special token.
Token	getNextToken() Returns the next token.
protected Regex[]	getRegexes() Returns the array of regexes (creating it if missing).
protected void	goobleWhiteSpace() Gobble input characters until next non-whitespace or input end.
int	length() CharSequence method.
void	setInput(java.lang.CharSequence anInput) Sets the current tokenizer input.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

_charIndex

protected int _charIndex

_lineIndex

protected int _lineIndex

_lineStart

protected int _lineStart

Constructor Detail

Tokenizer

public Tokenizer()

Method Detail

getInput

public java.lang.CharSequence getInput()

Returns the current tokenizer input.

setInput

public void setInput(java.lang.CharSequence anInput)

Sets the current tokenizer input.

getInput

public java.lang.CharSequence getInput(int aStart,
                                       int anEnd)

Returns the input subsequence for the given range of characters in input.

addPattern

public void addPattern(java.lang.String aName,
                       java.lang.String aPattern,
                       boolean isLiteral)

Adds a pattern.

addPatterns

public void addPatterns(ParseRule aRule)

Adds patterns to this tokenizer for given rule.

getRegexes

protected Regex[] getRegexes()

Returns the array of regexes (creating it if missing).

charAt

public char charAt(int anIndex)

CharSequence method.

length

public int length()

CharSequence method.

getChar

public char getChar()

Returns the current parse char.

getCharIndex

public int getCharIndex()

Returns the current parse char location.

getLineIndex

public int getLineIndex()

Returns the current line index.

getLineStart

public int getLineStart()

Returns the current line start index.

getColumnIndex

public int getColumnIndex()

Returns the current column index in the current line.

getLineNumber

public int getLineNumber()

Returns the current line number.

getColumnNumber

public int getColumnNumber()

Returns the current column number.

getNextToken

public Token getNextToken()

Returns the next token.

getMatchers

public Regex[] getMatchers(char aChar)

Returns list of matchers for a char.

createToken

protected Token createToken(java.lang.String aName,
                            java.lang.String aPattern,
                            int aStart,
                            int anEnd,
                            Token aSpclTkn)

Creates a new token.

getNextSpecialToken

public Token getNextSpecialToken()

Processes and returns next special token.

goobleWhiteSpace

protected void goobleWhiteSpace()

Gobble input characters until next non-whitespace or input end.