Plucene::Analysis::Tokenizer3pm
Base class for tokenizers
Usage
my $tokenizer = Plucene::Analysis::Tokenizer::Subclass->new({ reader => $reader });
Overview
This is an abstract base class for tokenizers.
A Tokenizer is a TokenStream whose input is a Reader.
Methods
new
my $tokenizer = Plucene::Analysis::Tokenizer::Subclass->new({ reader => $reader });
This will create a new tokenizer.
next
This must be defined in a subclass
close
Close the input reader.
Releases
11 releases since Debian 3.1 (Sarge); the text changed 5 times.