Post

Institutional Books is expanding. The Institutional Data Initiative @institutional.org at @harvardlawadmin.bsky.social Library has added paragraph-level text analysis and 22M extracted images to our work on ∼1M books, opening new possibilities for research, experimentation, and discovery. 🧵

Enriched Text contains 217B tokens from 983K @harvardlibrary.bsky.social volumes, organized into 1.39B annotated paragraphs across 250 languages. Researchers can use the dataset to analyze language, repeated passages, line breaks, and paragraph predictability.

Visual Elements extends the dataset to images embedded in scanned pages, producing 22.6M visual elements – including illustrations, photographs, charts, musical notation, and decorative materials – linked to their source volumes and pages.

Together, these projects establish Institutional Books as an ongoing resource that can grow through continued research and community use. The open-source pipelines can be adapted to new collections, models, languages, and questions: institutional.org/posts/extend...