Class Tokenize

All Implemented Interfaces:
Serializable, SourceLocator, LocationProvider, SaxonLocator, EvaluableItem, SequenceIterable, InstructionInfo, Locator

public class Tokenize extends SystemFunction
This class implements the tokenize() function for regular expression matching. This returns a sequence of strings representing the unmatched substrings: the separators which match the regular expression are not returned.
See Also:
  • Constructor Details

    • Tokenize

      public Tokenize()
  • Method Details

    • simplify

      public Expression simplify(ExpressionVisitor visitor) throws XPathException
      Simplify and validate. This is a pure function so it can be simplified in advance if the arguments are known
      Overrides:
      simplify in class FunctionCall
      Parameters:
      visitor - an expression visitor
      Returns:
      the simplified expression
      Throws:
      XPathException - if an error is discovered during expression rewriting
    • optimize

      public Expression optimize(ExpressionVisitor visitor, ItemType contextItemType) throws XPathException
      Perform optimisation of an expression and its subexpressions.

      This method is called after all references to functions and variables have been resolved to the declaration of the function or variable, and after all type checking has been done.

      Overrides:
      optimize in class SystemFunction
      Parameters:
      visitor - an expression visitor
      contextItemType - the static type of "." at the point where this expression is invoked. The parameter is set to null if it is known statically that the context item will be undefined. If the type of the context item is not known statically, the argument is set to Type.ITEM_TYPE
      Returns:
      the original expression, rewritten if appropriate to optimize execution
      Throws:
      XPathException - if an error is discovered during this phase (typically a type error)
    • getCompiledRegularExpression

      public RegularExpression getCompiledRegularExpression()
      Get the compiled regular expression if available, otherwise return null
      Returns:
      the compiled regular expression, or null
    • iterate

      public SequenceIterator iterate(XPathContext c) throws XPathException
      Iterate over the results of the function
      Specified by:
      iterate in interface SequenceIterable
      Overrides:
      iterate in class Expression
      Parameters:
      c - supplies the context for evaluation
      Returns:
      a SequenceIterator that can be used to iterate over the result of the expression
      Throws:
      XPathException - if any dynamic error occurs evaluating the expression
    • main

      public static void main(String[] args) throws Exception
      Simple command-line interface for testing.
      Parameters:
      args - (1) the string to be tokenized (2) the regular expression
      Throws:
      Exception