Package com.inductiveautomation.snap.parse

Class Tokenizer

  • Direct Known Subclasses:
    JavaTokenizer
    public class Tokenizer
extends java.lang.Object
    A class to extract tokens from a char sequence.

    • Constructor Summary

      Constructors 
      Constructor Description
      Tokenizer()  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void addPattern​(java.lang.String aName, java.lang.String aPattern, boolean isLiteral)
      Adds a pattern.
      void addPatterns​(ParseRule aRule)
      Adds patterns to this tokenizer for given rule.
      char charAt​(int anIndex)
      CharSequence method.
      protected Token createToken​(java.lang.String aName, java.lang.String aPattern, int aStart, int anEnd, Token aSpclTkn)
      Creates a new token.
      char getChar()
      Returns the current parse char.
      int getCharIndex()
      Returns the current parse char location.
      int getColumnIndex()
      Returns the current column index in the current line.
      int getColumnNumber()
      Returns the current column number.
      java.lang.CharSequence getInput()
      Returns the current tokenizer input.
      java.lang.CharSequence getInput​(int aStart, int anEnd)
      Returns the input subsequence for the given range of characters in input.
      int getLineIndex()
      Returns the current line index.
      int getLineNumber()
      Returns the current line number.
      int getLineStart()
      Returns the current line start index.
      Regex[] getMatchers​(char aChar)
      Returns list of matchers for a char.
      Token getNextSpecialToken()
      Processes and returns next special token.
      Token getNextToken()
      Returns the next token.
      protected Regex[] getRegexes()
      Returns the array of regexes (creating it if missing).
      protected void goobleWhiteSpace()
      Gobble input characters until next non-whitespace or input end.
      int length()
      CharSequence method.
      void setInput​(java.lang.CharSequence anInput)
      Sets the current tokenizer input.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • _charIndex

        protected int _charIndex

      • _lineIndex

        protected int _lineIndex

      • _lineStart

        protected int _lineStart

    • Constructor Detail

      • Tokenizer

        public Tokenizer()

    • Method Detail

      • getInput

        public java.lang.CharSequence getInput()
        Returns the current tokenizer input.

      • setInput

        public void setInput​(java.lang.CharSequence anInput)
        Sets the current tokenizer input.

      • getInput

        public java.lang.CharSequence getInput​(int aStart,
                                       int anEnd)
        Returns the input subsequence for the given range of characters in input.

      • addPattern

        public void addPattern​(java.lang.String aName,
                       java.lang.String aPattern,
                       boolean isLiteral)
        Adds a pattern.

      • addPatterns

        public void addPatterns​(ParseRule aRule)
        Adds patterns to this tokenizer for given rule.

      • getRegexes

        protected Regex[] getRegexes()
        Returns the array of regexes (creating it if missing).

      • charAt

        public char charAt​(int anIndex)
        CharSequence method.

      • length

        public int length()
        CharSequence method.

      • getChar

        public char getChar()
        Returns the current parse char.

      • getCharIndex

        public int getCharIndex()
        Returns the current parse char location.

      • getLineIndex

        public int getLineIndex()
        Returns the current line index.

      • getLineStart

        public int getLineStart()
        Returns the current line start index.

      • getColumnIndex

        public int getColumnIndex()
        Returns the current column index in the current line.

      • getLineNumber

        public int getLineNumber()
        Returns the current line number.

      • getColumnNumber

        public int getColumnNumber()
        Returns the current column number.

      • getNextToken

        public Token getNextToken()
        Returns the next token.

      • getMatchers

        public Regex[] getMatchers​(char aChar)
        Returns list of matchers for a char.

      • createToken

        protected Token createToken​(java.lang.String aName,
                            java.lang.String aPattern,
                            int aStart,
                            int anEnd,
                            Token aSpclTkn)
        Creates a new token.

      • getNextSpecialToken

        public Token getNextSpecialToken()
        Processes and returns next special token.

      • goobleWhiteSpace

        protected void goobleWhiteSpace()
        Gobble input characters until next non-whitespace or input end.