public class JavaTokenizer extends Tokenizer
_charIndex, _lineIndex, _lineStart
Constructor and Description |
---|
JavaTokenizer() |
Modifier and Type | Method and Description |
---|---|
Token |
getMultiLineCommentToken(Token aSpclTkn)
Process and return a multi-line comment if next up in input.
|
Token |
getMultiLineCommentTokenMore(Token aSpclTkn)
Returns a token from the current char to multi-line comment termination or input end.
|
Token |
getNextSpecialToken()
Processes and returns next special token.
|
Token |
getNextSpecialTokenImpl(Token aSpclTkn)
Processes and returns next special token.
|
Token |
getSingleLineCommentToken(Token aSpclTkn)
Processes and returns a single line comment token if next up in input.
|
addPattern, addPatterns, charAt, createToken, getChar, getCharIndex, getColumnIndex, getColumnNumber, getInput, getInput, getLineIndex, getLineNumber, getLineStart, getMatchers, getNextToken, getRegexes, goobleWhiteSpace, length, setInput
public Token getNextSpecialToken()
getNextSpecialToken
in class Tokenizer
public Token getNextSpecialTokenImpl(Token aSpclTkn)
public Token getSingleLineCommentToken(Token aSpclTkn)
public Token getMultiLineCommentToken(Token aSpclTkn)