edu.stanford.nlp.process
Class PTBTokenizer.PTBTokenizerFactory

java.lang.Object
  edu.stanford.nlp.process.PTBTokenizer.PTBTokenizerFactory

All Implemented Interfaces:

IteratorFromReaderFactory, TokenizerFactory

Enclosing class:

PTBTokenizer

public static class PTBTokenizer.PTBTokenizerFactory
extends Object
implements TokenizerFactory

Field Summary

protected boolean

tokenizeCRs

Constructor Summary

PTBTokenizer.PTBTokenizerFactory()
Constructs a new PTBTokenizerFactory that treats carriage returns as normal whitespace.

PTBTokenizer.PTBTokenizerFactory(boolean tokenizeCRs)
Constructs a new PTBTokenizer that optionally returns carriage returns as their own token.

Method Summary

Iterator	getIterator(Reader r)
Tokenizer	getTokenizer(Reader r)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

tokenizeCRs

protected boolean tokenizeCRs

Constructor Detail

PTBTokenizer.PTBTokenizerFactory

public PTBTokenizer.PTBTokenizerFactory()

Constructs a new PTBTokenizerFactory that treats carriage returns as normal whitespace.

PTBTokenizer.PTBTokenizerFactory

public PTBTokenizer.PTBTokenizerFactory(boolean tokenizeCRs)

Constructs a new PTBTokenizer that optionally returns carriage returns as their own token. CRs come back as Words whose text is the value of PTBLexer.cr.

Method Detail

getIterator

public Iterator getIterator(Reader r)

Specified by:

getIterator in interface IteratorFromReaderFactory

getTokenizer

public Tokenizer getTokenizer(Reader r)

Specified by:

getTokenizer in interface TokenizerFactory