protected static class StatefulTokenizer.Rule extends Object
getToken(String,int)
can be used to process input
data: If the rule matches a token is returned, otherwise null
will be returned.Constructor and Description |
---|
StatefulTokenizer.Rule(String pattern,
Object tokenType)
Initializes a new instance with the specified regular expression
pattern, a type for generated tokens.
|
StatefulTokenizer.Rule(String pattern,
Object tokenType,
String nextState)
Initializes a new instance with the specified regular expression
pattern, a type for generated tokens, and a name of a grammar state
that should be triggered.
|
Modifier and Type | Method and Description |
---|---|
StatefulTokenizer.Token |
getToken(String data,
int pos)
Analyzes the specified input data starting at the given position
and returns a token with the defined type, the content matched by
the regular expression if the rule matches.
|
public StatefulTokenizer.Rule(String pattern, Object tokenType, String nextState)
pattern
- A regular expression pattern string.tokenType
- The type for the tokens generated by this rule.nextState
- The grammar state that should be used next to
analyze the input data if the rule matched.public StatefulTokenizer.Rule(String pattern, Object tokenType)
pattern
- A regular expression pattern string.tokenType
- The type for the tokens generated by this rule.public StatefulTokenizer.Token getToken(String data, int pos)
null
will be returned.data
- Input data.pos
- Position to start looking for a match.null
if the rule didn't match.Copyright © 2009-2013. All Rights Reserved.