eu.cqse.check.framework.shallowparser

Class TokenStreamTextUtils

java.lang.Object

eu.cqse.check.framework.shallowparser.TokenStreamTextUtils

public class TokenStreamTextUtils
extends java.lang.Object

Utility methods for IToken lists that work on the tokens' text.

Constructor Summary

Constructors

Constructor and Description
TokenStreamTextUtils()

Method Summary

Modifier and Type	Method and Description
static java.util.List<java.lang.String>	concatAllTokenTexts(java.util.List<java.util.List<IToken>> tokenLists) Concatenates the inner lists' token texts and returns them as list.
static java.lang.String	concatTokenTexts(java.util.List<IToken> tokens) Concatenates the token's texts and returns them as string.
static java.lang.String	concatTokenTexts(java.util.List<IToken> tokens, java.lang.String separator) Concatenates the token's texts and returns them as string.
static boolean	contains(java.util.List<IToken> tokens, java.lang.String text) Returns whether the given token list contains a token with the given text.
static boolean	containsAny(java.util.List<IToken> tokens, java.util.Set<java.lang.String> texts) Returns whether the given token list contains a token whose text equals one of the texts from the given Set.
static java.util.List<java.lang.Integer>	findAll(java.util.List<IToken> tokens, java.lang.String text) Returns all indices of the given text within the given token list.
static java.util.List<java.lang.Integer>	findAllSequences(java.util.List<IToken> tokens, int startOffset, ETokenType startType, java.lang.String... sequence) Returns all indices of occurrences of the given sequence of token texts in the given token list, beginning from the given offset.
static int	findFirst(java.util.List<IToken> tokens, int startOffset, int endOffset, java.util.Set<java.lang.String> texts) Returns the first index of the first token not before the given start index and after the given end index whose text equals one of the given texts from the Set.
static int	findFirst(java.util.List<IToken> tokens, int startOffset, int endOffset, java.lang.String text) Returns the first index of the given text not before the given start index and before the given end index.
static int	findFirst(java.util.List<IToken> tokens, java.util.Set<java.lang.String> texts) Returns the index of the first token whose token text equals one of the given texts from the Set.
static int	findFirst(java.util.List<IToken> tokens, java.lang.String text) Returns the index of the first token whose token text equals the given text in the given token list.
static int	findFirstCaseInsensitive(java.util.List<IToken> tokens, int startOffset, int endOffset, java.lang.String text) Returns the first index of the given text not before the given start index and before the given end index.
static int	findSequence(java.util.List<IToken> tokens, int startOffset, ETokenType startType, java.lang.String... sequence) Returns the index of the first occurrence of the given sequence of strings in the given token list, beginning from the given start offset.
static java.util.List<java.lang.String>	getTokenTexts(java.util.List<IToken> tokens) Converts the given token list to a list of corresponding token texts.
static java.util.List<java.lang.String>	getTokenTexts(java.util.List<IToken> tokens, int startIndex, int endIndex) Converts the sublist of the given token list from the given start index (inclusive) to the given end index (exclusive) to a list of token texts.
static boolean	hasSequence(java.util.List<IToken> tokens, int startOffset, java.lang.String... sequence) Returns whether the given token list contains tokens with the given sequence of strings at the given start offset.
static boolean	is(IToken token, ETokenType tokenType, java.lang.String tokenText) Returns whether the given token has the given type and text.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TokenStreamTextUtils

public TokenStreamTextUtils()

Method Detail

findFirst

public static int findFirst(java.util.List<IToken> tokens,
                            java.lang.String text)

Returns the index of the first token whose token text equals the given text in the given token list. If the text is not found, #NOT_FOUND is returned.

findFirst

public static int findFirst(java.util.List<IToken> tokens,
                            java.util.Set<java.lang.String> texts)

Returns the index of the first token whose token text equals one of the given texts from the Set. If none of the texts are found, #NOT_FOUND is returned.

findFirst

public static int findFirst(java.util.List<IToken> tokens,
                            int startOffset,
                            int endOffset,
                            java.lang.String text)

Returns the first index of the given text not before the given start index and before the given end index. If the text is not found, #NOT_FOUND is returned.

findFirst

public static int findFirst(java.util.List<IToken> tokens,
                            int startOffset,
                            int endOffset,
                            java.util.Set<java.lang.String> texts)

Returns the first index of the first token not before the given start index and after the given end index whose text equals one of the given texts from the Set. If none of the texts are found, #NOT_FOUND is returned.

findFirstCaseInsensitive

public static int findFirstCaseInsensitive(java.util.List<IToken> tokens,
                                           int startOffset,
                                           int endOffset,
                                           java.lang.String text)

Returns the first index of the given text not before the given start index and before the given end index. If the text is not found, #NOT_FOUND is returned.

findAll

public static java.util.List<java.lang.Integer> findAll(java.util.List<IToken> tokens,
                                                        java.lang.String text)

Returns all indices of the given text within the given token list. If none is found, an empty list is returned.

contains

public static boolean contains(java.util.List<IToken> tokens,
                               java.lang.String text)

Returns whether the given token list contains a token with the given text.

containsAny

public static boolean containsAny(java.util.List<IToken> tokens,
                                  java.util.Set<java.lang.String> texts)

Returns whether the given token list contains a token whose text equals one of the texts from the given Set.

hasSequence

public static boolean hasSequence(java.util.List<IToken> tokens,
                                  int startOffset,
                                  java.lang.String... sequence)

Returns whether the given token list contains tokens with the given sequence of strings at the given start offset.

findSequence

public static int findSequence(java.util.List<IToken> tokens,
                               int startOffset,
                               ETokenType startType,
                               java.lang.String... sequence)

Returns the index of the first occurrence of the given sequence of strings in the given token list, beginning from the given start offset. The given start type is the type of the sequence's first token. If the sequence is not found, #NOT_FOUND is returned.

findAllSequences

public static java.util.List<java.lang.Integer> findAllSequences(java.util.List<IToken> tokens,
                                                                 int startOffset,
                                                                 ETokenType startType,
                                                                 java.lang.String... sequence)

Returns all indices of occurrences of the given sequence of token texts in the given token list, beginning from the given offset. The given start token type is the type of the first token of the sequence. If the sequence is not found, an empty list is returned.

concatTokenTexts

public static java.lang.String concatTokenTexts(java.util.List<IToken> tokens)

Concatenates the token's texts and returns them as string.

concatTokenTexts

public static java.lang.String concatTokenTexts(java.util.List<IToken> tokens,
                                                java.lang.String separator)

Concatenates the token's texts and returns them as string.

concatAllTokenTexts

public static java.util.List<java.lang.String> concatAllTokenTexts(java.util.List<java.util.List<IToken>> tokenLists)

Concatenates the inner lists' token texts and returns them as list.

getTokenTexts

public static java.util.List<java.lang.String> getTokenTexts(java.util.List<IToken> tokens,
                                                             int startIndex,
                                                             int endIndex)

Converts the sublist of the given token list from the given start index (inclusive) to the given end index (exclusive) to a list of token texts.

getTokenTexts

public static java.util.List<java.lang.String> getTokenTexts(java.util.List<IToken> tokens)

Converts the given token list to a list of corresponding token texts.

is

public static boolean is(IToken token,
                         ETokenType tokenType,
                         java.lang.String tokenText)

Returns whether the given token has the given type and text. The text comparison is case insensitive if the token's language is case insensitive.