edu.stanford.nlp.trees
Class PennTreebankTokenizer
java.lang.Object
edu.stanford.nlp.process.AbstractTokenizer
edu.stanford.nlp.process.TokenizerAdapter
edu.stanford.nlp.trees.PennTreebankTokenizer
- All Implemented Interfaces:
- Tokenizer, Iterator
public class PennTreebankTokenizer
- extends TokenizerAdapter
Builds a tokenizer for English PennTreebank (release 2) trees.
This is currently internally implemented via a java.io.StreamTokenizer.
- Author:
- Christopher Manning, Roger Levy
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
PennTreebankTokenizer
public PennTreebankTokenizer(Reader r)
Stanford NLP Group