edu.stanford.nlp.process
Class WordSegmentingTokenizer

java.lang.Object
  edu.stanford.nlp.process.AbstractTokenizer
      edu.stanford.nlp.process.WordSegmentingTokenizer

All Implemented Interfaces:

Tokenizer, Iterator

public class WordSegmentingTokenizer
extends AbstractTokenizer

Author:

Galen Andrew

Field Summary

Fields inherited from class edu.stanford.nlp.process.AbstractTokenizer

nextToken

Constructor Summary

WordSegmentingTokenizer(WordSegmenter wordSegmenter, Reader r)

Method Summary

static TokenizerFactory	factory(WordSegmenter wordSegmenter)
protected Object	getNext() Internally fetches the next token.
Sentence	segmentWords(String s)

Methods inherited from class edu.stanford.nlp.process.AbstractTokenizer

hasNext, next, peek, remove, tokenize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WordSegmentingTokenizer

public WordSegmentingTokenizer(WordSegmenter wordSegmenter,
                               Reader r)

Method Detail

getNext

protected Object getNext()

Description copied from class: AbstractTokenizer

Internally fetches the next token.

Specified by:

getNext in class AbstractTokenizer

Returns:

the next token in the token stream, or null if none exists.

segmentWords

public Sentence segmentWords(String s)

factory

public static TokenizerFactory factory(WordSegmenter wordSegmenter)