|
||||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object | +--org.argouml.util.MyTokenizer
Class for dividing a String into any number of parts. Each part will be a substring of the original String. The first part will at least contain the first character in the string. All following parts will at least contain the first character in the String not covered by any previous part.
The delim parameter to the constructors is a comma separated list of tokens that should be recognized by the tokenizer. These tokens will be returned by the tokenizer as tokens, and any arbitrary text between them will also be returned as tokens. Since the comma has special meaning in this string, it can be escaped with \ to only mean itself (like in "\\,"). For technical reasons it is not possible for any token in this list to be more than 32 characters long.
In addition to the delim parameter it is also possible to use custom separators that allow any string that can be generated by the limited version of a Turing machine that your computer is, to be used as a delimiter.
There are some custom separators provided that you can use to get things like strings in one token. These cannot be used simultaneously by several tokenizers, ie they are not thread safe.
The tokenizer works in a kind of greedy way. When the first separator token from delim is matched or any CustomSeparator returns true from addChar, then it is satisfied it has found a token and does NOT check if it could have found a longer token. Eg: if you have this delim string "<,<<", then "<<" will never be found.
Example
MyTokenizer tzer = new MyTokenizer("Hello, how are you?", " ,\\,"); while (tzer.hasMoreTokens()) _cat.info("\"" + tzer.nextToken() + "\"");
Which whould yield the following output:
"Hello" "," " " "how" " " "are" " " "you?"
CustomSeparator
Field Summary | |
---|---|
private java.util.Vector |
_customSeps
|
private TokenSep |
_delims
|
private int |
_eIdx
|
private java.lang.String |
_putToken
|
private int |
_savedIdx
|
private java.lang.String |
_savedToken
|
private int |
_sIdx
|
private java.lang.String |
_source
|
private int |
_tokIdx
|
static CustomSeparator |
DOUBLE_QUOTED_SEPARATOR
A custom separator for quoted strings enclosed in double quotes and using \ as escape character. |
static CustomSeparator |
PAREN_EXPR_SEPARATOR
A custom separator for expressions enclosed in parentheses and matching lparams with rparams. |
static CustomSeparator |
PAREN_EXPR_STRING_SEPARATOR
A custom separator for expressions enclosed in parentheses and matching lparams with rparams. |
static CustomSeparator |
SINGLE_QUOTED_SEPARATOR
A custom separator for quoted strings enclosed in single quotes and using \ as escape character. |
Constructor Summary | |
---|---|
MyTokenizer(java.lang.String string,
java.lang.String delim)
Constructs a new instance. |
|
MyTokenizer(java.lang.String string,
java.lang.String delim,
java.util.Collection seps)
Constructs a new instance. |
|
MyTokenizer(java.lang.String string,
java.lang.String delim,
CustomSeparator sep)
Constructs a new instance. |
Method Summary | |
---|---|
int |
getTokenIndex()
Returns the index in the string of the last token returned by nextToken, or zero if no token has been retrived. |
boolean |
hasMoreElements()
This class implements the Enumeration interface. |
boolean |
hasMoreTokens()
Returns true if there are more tokens left. |
java.lang.Object |
nextElement()
This class implements the Enumeration interface. |
java.lang.String |
nextToken()
Retrives the next token. |
private static TokenSep |
parseDelimString(java.lang.String str)
Creates a linked list of TokenSeps from the comma separated string str. |
void |
putToken(java.lang.String s)
Put a token on the input stream. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
public static final CustomSeparator SINGLE_QUOTED_SEPARATOR
public static final CustomSeparator DOUBLE_QUOTED_SEPARATOR
public static final CustomSeparator PAREN_EXPR_SEPARATOR
public static final CustomSeparator PAREN_EXPR_STRING_SEPARATOR
private int _sIdx
private final int _eIdx
private int _tokIdx
private final java.lang.String _source
private final TokenSep _delims
private java.lang.String _savedToken
private int _savedIdx
private java.util.Vector _customSeps
private java.lang.String _putToken
Constructor Detail |
public MyTokenizer(java.lang.String string, java.lang.String delim)
string
- The String to be tokenized.delim
- The String of delimiters.public MyTokenizer(java.lang.String string, java.lang.String delim, CustomSeparator sep)
string
- The String to be tokenized.delim
- The String of delimiters.sep
- A custom separator to use.public MyTokenizer(java.lang.String string, java.lang.String delim, java.util.Collection seps)
string
- The String to be tokenized.delim
- The String of delimiters.seps
- Some container with custom separators to use.Method Detail |
public boolean hasMoreTokens()
public java.lang.String nextToken()
public java.lang.Object nextElement()
nextElement
in interface java.util.Enumeration
nextToken
public boolean hasMoreElements()
hasMoreElements
in interface java.util.Enumeration
hasMoreTokens
public int getTokenIndex()
public void putToken(java.lang.String s)
The index returned from getTokenIndex will be the same for the token put as that of the last token that wasn't put.
s
- The token to put.
java.lang.NullPointerException
- if s is null.private static TokenSep parseDelimString(java.lang.String str)
str
- The string specifying delimiter strings.
|
||||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
ArgoUML © 1996-2004 (20040306) | ArgoUML Homepage | ArgoUML Developers' page | ArgoUML Cookbook |