I am very excited to release WordLevel, an open source Python package for comparing texts at the word level! wordlevel.dev WordLevel makes it easy to compare texts with measures ranging from the Jensen-Shannon divergence to dictionary-based sentiment, and visualize them as word shift graphs.
Post
WordLevel is designed to be fast and reliable. It has a modern Python interface with a declarative API powered by Polars. You can use it with any text that you can tokenize, and it integrates with any Narwhals-supported dataframe. The word shift graphs are publication ready and fully customizable.
0 reposts1 like
I’m excited to share WordLevel, which incorporates a lot of the lessons in software engineering, data analysis, and scientific computing that I have learned since Shifterator. If you want to learn more, dive into the docs or check out the code! Docs: wordlevel.dev Code: github.com/ryanjgallagh...
wordlevel.devwordlevelPython package for natural language processing at the word level
If you would like to learn more about word shift graphs, please read our paper! link.springer.com/article/10.1...
link.springer.comGeneralized word shift graphs: a method for visualizing and explaining pairwise comparisons between texts - EPJ Data ScienceA common task in computational text analyses is to quantify how two corpora differ according to a measurement like word frequency, sentiment, or information content. However, collapsing the texts’ ric...