Rank-frequency (Zipf)
Natural languages follow Zipf's law: a word's frequency falls roughly in inverse proportion to its rank, so on a log-log plot the curve is close to a straight line. The Voynich corpus is compared with equally sized reference corpora: Hebrew (WLC, Mishnah, Mishneh Torah), Aramaic (Targum Onkelos) and Arabic in Arabic and in Hebrew script. Controls are non-Semitic texts (Italian, Latin, Hawaiian, Greek) and the project's own pseudo-word generator; the Naibbe cipher's ciphertext stands alongside them as a competing cipher model. The test is necessary, not sufficient: a Zipf curve rules out simple random generators, but it does not identify a language.
Rank-frequency curves
Statistics
| Corpus | Tokens | Types | Type/token | Hapax share | α | R² |
|---|---|---|---|---|---|---|
| Voynich (all) | Loading corpus data… | |||||
| Voynich Currier A | Loading corpus data… | |||||
| Voynich Currier B | Loading corpus data… | |||||
| Voynich Currier B, at A's size | Loading corpus data… | |||||
| WLC Hebrew | 37,786 | 10,204 | 0.270 | 62.1 % | 0.816 | 0.995 |
| Mishnah | 37,944 | 8,441 | 0.222 | 53.2 % | 0.831 | 0.993 |
| Mishneh Torah | 37,928 | 9,678 | 0.255 | 58.9 % | 0.839 | 0.992 |
| Targum Onkelos | 38,000 | 8,432 | 0.222 | 55.2 % | 0.840 | 0.997 |
| Arabic (Ibn al-Bayṭār) | 38,000 | 9,412 | 0.248 | 61.0 % | 0.825 | 0.996 |
| Judeo-Arabic (Maimonides) | 38,000 | 8,587 | 0.226 | 62.2 % | 0.906 | 0.990 |
| Dante, Commedia (Ital.) | 38,000 | 7,463 | 0.196 | 62.4 % | 0.995 | 0.995 |
| Boccaccio, Decameron (Ital.) | 38,000 | 5,809 | 0.153 | 57.2 % | 1.005 | 0.994 |
| Caesar, De bello Gallico (Lat.) | 38,000 | 9,060 | 0.238 | 58.3 % | 0.843 | 0.994 |
| Virgil, Aeneid (Lat.) | 38,000 | 12,329 | 0.324 | 59.8 % | 0.695 | 0.994 |
| Gesta Romanorum (Med. Lat.) | 38,000 | 7,037 | 0.185 | 53.6 % | 0.874 | 0.994 |
| Hawaiian Bible (1868) | 37,649 | 1,752 | 0.047 | 41.9 % | 1.342 | 0.991 |
| Dioscorides (Greek) | 38,000 | 8,844 | 0.233 | 63.6 % | 0.906 | 0.996 |
| Generator control | Loading corpus data… | |||||
| Naibbe ciphertext (Greshko 2025) | 34,764 | 5,600 | 0.161 | 40.9 % | 0.922 | 0.974 |
Method: every consensus token counts (no deduplication). Reference corpora: consonantal words as written (proclitics attached, niqqud removed), about 38,000 tokens each, taken as equal blocks from the middle of every book or tractate. Control texts of the same size: Latin script as lower-case letter runs (in Italian the elision apostrophe stays word-internal), Greek without accents and breathings. Macer Floridus is left out because, at 13,967 tokens, it is smaller than the sample. α = negative slope of the least-squares line of ln(frequency) on ln(rank) over ranks 1–1000, types with at least two occurrences only. Arabic without harakat, Judeo-Arabic without the diacritic geresh, Naibbe ciphertext as EVA tokens (34,764, complete). Control: as many independent pseudo-words from the Markov generator (seed 1) as the corpus has tokens. Currier B at A's size: random sample of B's tokens without replacement (seed 1), as many tokens as A. Voynich and control curves are computed live from the current transcription data.
Reproduction & sources
Download toolkit (.zip)Contains the download and analysis scripts for the reference curves, README and license notices. Only the Python standard library is needed; the reference corpora themselves are not included (downloaded by script, see README).
Reference corpora (13)
- Westminster Leningrad Codex (WLC) · Text: Public Domain · Annotation: CC BY 4.0
- Mishnah, Vilna edition 1913 (Sefaria export) · Public Domain
- Maimonides, Mishneh Torah (Wikisource, via the Sefaria export) · CC BY-SA
- Targum Onkelos, Yemenite Taj vocalization (Wikisource, via the Sefaria export) · CC BY-SA
- Maimonides, Guide for the Perplexed, Judeo-Arabic original, Munk edition, Paris 1856 (Sefaria export) · Public Domain
- Ibn al-Bayṭār, al-Jāmiʿ li-mufradāt al-adwiya wa-l-aghdhiya (OpenITI) · CC BY-NC-SA 4.0
- Dante, Divina Commedia (Wikisource) · CC BY-SA 4.0
- Boccaccio, Decameron, Vittore Branca edition (Biblioteca italiana, Sapienza Rome) · Public Domain
- Virgil, Aeneid / Caesar, De bello Gallico (Perseus Digital Library) · CC BY-SA 4.0
- Macer Floridus, De viribus herbarum / Gesta Romanorum, Oesterley edition 1872 (Latin Wikisource) · CC BY-SA 4.0
- Baibala Hemolele, Hawaiian Bible 1868 (eBible.org) · Public Domain
- Dioscorides, De materia medica, Wellmann edition 1906–1914 (First1KGreek) · CC BY-SA 4.0
- Naibbe cipher, ciphertext of Pliny, Naturalis historia (Greshko 2025, Cryptologia) · MIT (modified, citation required)