Previous Topic: Functions for Splitting StringsNext Topic: GetToken


SetTokenizerInput

The SetTokenizerInput function initializes the tokenizer with a string and defines the delimiters for splitting the string. This string and delimiters are valid for the rest of the script until you call the function with a different string or delimiter.

Function format:

SetTokenizerInput(str as string, delimiters as string)

Input Parameters

This function has the following input parameters:

str

Specifies the input string that you want to split.

Note: You can specify only one input string at a time.

delimiters

Specifies the set of delimiters for splitting the string. By default, space characters are used as delimiters, but you can specify any set of characters. For example, “.,/” tokenizes a list of substrings separated by a dot, comma or slash.

Return Values

None