nltk.stem.porter module

Porter Stemmer

This is the Porter stemming algorithm. It follows the algorithm presented in

Porter, M. “An algorithm for suffix stripping.” Program 14.3 (1980): 130-137.

with some optional deviations that can be turned on or off with the mode argument to the constructor.

Martin Porter, the algorithm’s inventor, maintains a web page about the algorithm at

https://www.tartarus.org/~martin/PorterStemmer/

which includes another Python implementation and other implementations in many languages.

class nltk.stem.porter.PorterStemmer

Bases: StemmerI

A word stemmer based on the Porter stemming algorithm.

Porter, M. “An algorithm for suffix stripping.” Program 14.3 (1980): 130-137.

See https://www.tartarus.org/~martin/PorterStemmer/ for the homepage of the algorithm.

Martin Porter has endorsed several modifications to the Porter algorithm since writing his original paper, and those extensions are included in the implementations on his website. Additionally, others have proposed further improvements to the algorithm, including NLTK contributors. There are thus three modes that can be selected by passing the appropriate constant to the class constructor’s mode attribute:

• PorterStemmer.ORIGINAL_ALGORITHM

  An implementation that is faithful to the original paper.

  Note that Martin Porter has deprecated this version of the algorithm. Martin distributes implementations of the Porter Stemmer in many languages, hosted at:

  https://www.tartarus.org/~martin/PorterStemmer/

  and all of these implementations include his extensions. He strongly recommends against using the original, published version of the algorithm; only use this mode if you clearly understand why you are choosing to do so.

• PorterStemmer.MARTIN_EXTENSIONS

  An implementation that only uses the modifications to the algorithm that are included in the implementations on Martin Porter’s website. He has declared Porter frozen, so the behaviour of those implementations should never change.

• PorterStemmer.NLTK_EXTENSIONS (default)

  An implementation that includes further improvements devised by NLTK contributors or taken from other modified implementations found on the web.

For the best stemming, you should use the default NLTK_EXTENSIONS version. However, if you need to get the same results as either the original algorithm or one of Martin Porter’s hosted versions for compatibility with an existing implementation or dataset, you can use one of the other modes instead.

MARTIN_EXTENSIONS = 'MARTIN_EXTENSIONS'

NLTK_EXTENSIONS = 'NLTK_EXTENSIONS'

ORIGINAL_ALGORITHM = 'ORIGINAL_ALGORITHM'

__init__(mode='NLTK_EXTENSIONS')

stem(word, to_lowercase=True)

Parameters

to_lowercase – if to_lowercase=True the word always lowercase

nltk.stem.porter.demo()

A demonstration of the porter stemmer on a sample from the Penn Treebank corpus.