Vocabulary Richness Calculator

Lexical diversity — how varied a text’s vocabulary is — is used in language teaching, child-language research, authorship studies and readability work. The idea is simple: count how many different words a text uses relative to its length. The details are not, because nearly every simple measure changes with text length. This calculator gives the classic measures alongside the modern length-robust ones, so you can compare texts fairly.

Tokens, types and hapax

Tokens are all the words in the text; types are the different words. “The cat and the dog” has five tokens and four types, because “the” appears twice. Hapax legomena are words used only once. Words are compared without regard to capital letters, and different forms of a word, such as walk and walked, count as different types because the tool does not reduce words to dictionary forms.

Why the type-token ratio misleads

The type-token ratio (TTR) divides types by tokens. It works for texts of identical length but falls steadily as a text grows, because every text keeps reusing its common words. A 100-word paragraph and a 10,000-word chapter by the same author can differ hugely in TTR. Root TTR (Guiraud), corrected TTR (Carroll) and Herdan’s C apply mathematical corrections that reduce, but do not remove, this effect.

MTLD and HD-D

MTLD (Measure of Textual Lexical Diversity) walks through the text and counts how many words pass before the running TTR drops to 0.72, then averages that run length over the text, reading it forwards and backwards. HD-D uses the hypergeometric distribution to estimate the TTR of a random 42-word sample of the text. Both were shown by McCarthy and Jarvis to be largely independent of length, which makes them the measures of choice for comparing texts of different sizes. MTLD needs at least 50 words and HD-D at least 42.

Yule’s K

Yule’s K measures how often words repeat, using the whole frequency distribution. Lower values mean a richer vocabulary. It has a long history in stylometry and authorship studies.

Making fair comparisons

Compare texts of similar length and genre, use at least a few hundred words each, and prefer MTLD or HD-D. Clean the texts the same way — remove headers, page numbers and quotations from other authors — before measuring.

A worked example

The opening of A Tale of Two Cities pasted as the sample is famously repetitive: “it was the best of times, it was the worst of times…”. Its 85 words use only 34 different words, a type-token ratio of 0.40, which is low for such a short passage; the repeated frame “it was the … of …” accounts for most of the repetition. A paragraph of the same length from a nature essay full of specific nouns shows a higher MTLD and more words used once — two very different styles, both deliberate.

Using the results in teaching

For language learners, compare a student’s writing at the start and end of a course on similar tasks and lengths; a rising MTLD suggests a growing active vocabulary. Combine the numbers with a look at the word frequency list, which shows whether new words are varied or the same few repeated.

Related tools

See which words repeat with the Word Frequency Counter, check difficulty with the Readability Checker or the Dale–Chall Calculator, or count words with the Word Counter.