An n-gram is a contiguous sequence of n items from a given sample of text or speech. The items can be characters, words, or even phonemes. For example, in the sentence “the cat sat,” the 2-grams (bigrams) are “the cat” and “cat sat.” This calculator computes the frequency of n-grams in a text you provide, which is the first step in many natural language processing tasks such as language modeling, spell checking, and text classification.
Formula
The formula for counting n-grams is simple: given a sequence of tokens (words or characters) of length L, the number of n-grams is L – n + 1 (if no padding is used). Each n-gram is a contiguous subsequence of n tokens. If padding is applied, special tokens like <s> and </s> are added at the beginning and end, increasing the count accordingly.
For a text tokenized into words w1, w2, …, wL, the n-grams are:
w1…wn, w2…wn+1, …, wL-n+1…wL
Each n-gram’s frequency is the number of times it appears in the text.
Worked Example
Let’s use the default text: “the cat sat on the mat” with n=2 and no padding.
Step 1: Tokenize into words: [the, cat, sat, on, the, mat] (L=6).
Step 2: Generate all bigrams (n=2):
- the cat
- cat sat
- sat on
- on the
- the mat
Step 3: Count frequencies: each bigram appears exactly once, so each has frequency 1.
The calculator outputs the list of n-grams with their counts. For the default text, you get 5 bigrams, each with count 1.
Assumptions
- Tokens are separated by whitespace (words). For character n-grams, the text is split into individual characters.
- Case is preserved; “The” and “the” are different tokens unless you normalize the text first.
- Punctuation is treated as part of the token unless you remove it before input.
- The calculator counts overlapping n-grams (sliding window), not non-overlapping segments.
Background: Where n-grams come from
The term “n-gram” was coined by Claude Shannon in his 1948 paper “A Mathematical Theory of Communication,” where he used them to model the statistical structure of language. The idea was to estimate the probability of the next character or word given the previous n-1. Shannon’s work laid the foundation for information theory and statistical natural language processing.
In the 1980s, n-gram models became a core component of speech recognition and machine translation. Today, they are used in search engines (for autocomplete and spell correction), in text prediction on smartphones, and in computational linguistics for analyzing corpora.
How to choose the value of n
The choice of n depends on the task and the size of your corpus:
- n=1 (unigram): treats each token independently; used for bag-of-words models, sentiment analysis, and keyword extraction.
- n=2 (bigram): captures adjacent word pairs; useful for simple language models and collocation detection.
- n=3 (trigram): captures short context; commonly used in language modeling and text classification.
- n=4 to 6: used in specialized tasks like authorship attribution or when you need longer context, but they suffer from data sparsity.
For character n-grams, n=3 to 5 are typical for language identification and spelling correction. For word n-grams, n=2 or 3 is a good starting point for most tasks. Larger n values require exponentially more data to estimate probabilities reliably.
Common problems and misconceptions
One common mistake is not normalizing the text. If you don’t lowercase words or remove punctuation, “The” and “the” are counted as different tokens, which can skew frequency counts. For many applications, you should preprocess the text: convert to lowercase, remove punctuation, and possibly stem words.
Another issue is data sparsity: with large n, many n-grams appear only once or not at all in the training data. This leads to zero probabilities in language models, which is why smoothing techniques (like Laplace smoothing) are used. The calculator here just counts frequencies; it doesn’t apply smoothing.
Also, note that n-grams are contiguous. “The cat” and “the mat” share the word “the” but are different bigrams. Some people confuse n-grams with “skip-grams” (which allow gaps) or “bag of words” (which ignores order).
Applications of n-grams
N-grams are used in many practical areas:
- Spell checking: a misspelled word is often detected by looking at character n-grams that don’t match common patterns.
- Machine translation: n-gram language models help choose the most fluent translation.
- Sentiment analysis: bigrams like “not good” carry more meaning than individual words.
- Search engines: autocomplete and query suggestion rely on n-gram frequencies from large corpora.
- Authorship attribution: the frequency of function-word n-grams can help identify the author of a text.
In bioinformatics, n-grams (often called k-mers) are used to analyze DNA sequences, where n is typically 5 to 30.
Alternatives to n-grams
While n-grams are simple and effective, they have limitations: they don’t capture long-range dependencies or semantic similarity. Alternatives include:
- Bag-of-words (BOW): ignores word order entirely, using only word frequencies. Simpler but loses context.
- TF-IDF: weights words by importance in a document relative to a corpus; often used with n-grams.
- Word embeddings: like Word2Vec or GloVe, represent words as dense vectors that capture semantic meaning, but they require training on large corpora.
- Transformer models: like BERT, use attention mechanisms to model context without fixed n-gram windows, but are much more computationally intensive.
For many tasks, a well-tuned n-gram model still beats complex models, especially with limited data.
Frequently Asked Questions
What is the difference between a unigram, bigram, and trigram? A unigram is a single token (n=1), a bigram is a sequence of two tokens (n=2), and a trigram is a sequence of three tokens (n=3). They capture increasing amounts of local context.
How do I handle punctuation in n-gram counting? It depends on your goal. For word n-grams, you typically remove punctuation and lowercase all words. For character n-grams, you might keep punctuation as tokens. The calculator treats everything as separate tokens, so you should preprocess accordingly.
What is smoothing in n-gram models? Smoothing is a technique to assign non-zero probabilities to n-grams that never appear in the training data. Common methods include Laplace (add-one) smoothing, Kneser-Ney, and Good-Turing. This calculator only counts frequencies; it doesn’t build a probability model.
Can n-grams be used for non-English text? Yes, n-grams work for any language that can be tokenized into words or characters. For languages without spaces (like Chinese), character n-grams are often used.
Why do my n-gram counts seem too low? If you use a large n on a short text, the number of n-grams is small (L – n + 1). For example, a 10-word text with n=5 gives only 6 n-grams. Also, if you use padding, the count increases by n-1 (for start padding) and n (for end padding) depending on the option.
What is the difference between word n-grams and character n-grams? Word n-grams operate on words and capture syntactic patterns; character n-grams operate on individual characters and are robust to spelling variations and unknown words. Character n-grams are often used in language identification and spam filtering.
Related
Formula used
in t | Input text | - | the cat sat on the mat | 0 in n | n-gram size (n) | - | 2 | 1 in p | Padding | - | none=0, start=1, both=2 | 0 out f | Frequency of each n-gram | - | text | 0
Each line is one input (in, sel, list) or one result (out: label, unit, formula, decimals). How the calculators work
Method: the standard formula for this calculation with the constants and conversion factors written on the page, rounded as each result states. See method and constants, the disclaimer or report an error.
More in this section
- ug to ppb Calculator: Convert Micrograms per Liter1.00 ppb
- Grams to Moles Chemistry: How to Convert0.2000 mol/L
- Convert ug/l to ppb Calculator
- CPP Calculator Ontario: How CPP Is Calculated162.12 $/month
- Grams to Moles to Molecules: How to Convert0.2000 mol/L
- Organic Chemistry Reaction Calculator: How It Works