edu.stanford.nlp.process
Interface LexedTokenFactory

All Known Implementing Classes:

WordTokenFactory

public interface LexedTokenFactory

Constructs a token (of arbitrary type) from a String and its position in the underlying text.

Method Summary

Object

makeToken(String str, int begin, int length) Constructs a token (of arbitrary type) from a String and its position in the underlying text.

Method Detail

makeToken

Object makeToken(String str,
                 int begin,
                 int length)

Constructs a token (of arbitrary type) from a String and its position in the underlying text.

Parameters:

str - The String extracted by the lexer.

begin - The offset in the document of the first character in this string.

length - The number of characters the string takes up in the document.