Marzena Karpinska

Marzena Karpinska

@markar.bsky.social

#nlp researcher interested in evaluation including: multilingual models, long-form input/output, processing/generation of creative texts previous: postdoc @ umass_nlp phd from utokyo https://marzenakrp.github.io/

3.9K followers979 following122 posts

Marzena Karpinska reposted

Attending #EMNLP2026 or following along from home? Tired of Whova and Underline? Use infrastructure made by researchers instead! Lea recommends talks/posters for you; organizes your own presentations; shows you who is attending; includes a live feed of posts; and more! lea.ac/conferences/...

We still need help. If you can spare 30-45 min after #EMNLP to chat about your language experience in Hungary please reach out!

Ehud Reiter@ehudreiter.bsky.social

Going to EMNLP 2026 in Budapest? We are looking for 20-30 EMNLP attendees to help us understand how you handle language barriers during a conference trip. Contact Yujun Wang: y.wang2.25@abdn.ac.uk @weizhaonlp.bsky.social, @markar.bsky.social, Pinzhen Chen, Yujun Wang

Hello #NLP Anyone is going to #EMNLP and can help?

Ehud Reiter@ehudreiter.bsky.social

Going to EMNLP 2026 in Budapest? We are looking for 20-30 EMNLP attendees to help us understand how you handle language barriers during a conference trip. Contact Yujun Wang: y.wang2.25@abdn.ac.uk @weizhaonlp.bsky.social, @markar.bsky.social, Pinzhen Chen, Yujun Wang

Marzena Karpinska reposted

This is complex — and likely a bit fragile — but very cool. Actually less interested in the ostensible end goal (Pangram for Ideas) and more in the problems you have to solve to get there (separating ideas from style, and learning to recognize more/less predictable ideas).

Vauhini Vara@vauhinivara.bsky.social

Here's the paper that accompanies the demo. The researchers find that IdeaLens can fairly accurately flag (when Pangram can't) stories that are written by people based on "AI-generated plans."

Can you tell whether the #ideas in a piece of writing came from a human or an AI? As AI writing becomes common, policies on AI use focus on who came up with the ideas in the text rather than who wrote it. AI detectors answer the latter, not the former. We release #IdeaLens, a detector of AI ideas.

View full thread

Can you tell whether the #ideas in a piece of writing came from a human or an AI? As AI writing becomes common, policies on AI use focus on who came up with the ideas in the text rather than who wrote it. AI detectors answer the latter, not the former. We release #IdeaLens, a detector of AI ideas.

View full thread

IdeaLens can also detect AI ideas in languages other than English despite never seeing them during training. This also works in low-resource languages such as Marathi or Khmer. The model flags 94- 96% of AI-ideated documents & only 0.6-0.8% of AI-written documents.

We analyze some trends in AI ideas and Human ideas to understand what the detector is picking up on. We see that: - AI overstates claims beyond what is provided - Makes statements that 'appear' specific until you try to actually apply them.

Can you tell whether the #ideas in a piece of writing came from a human or an AI? As AI writing becomes common, policies on AI use focus on who came up with the ideas in the text rather than who wrote it. AI detectors answer the latter, not the former. We release #IdeaLens, a detector of AI ideas.

View full thread

IdeaLens can detect varying levels of human input that went into the writing. For documents generated from a very detailed human outline, IdeaLens flags only 6.8% as AI, compared to ProseLens's 86.5% and Pangram's 92.4%.

Can you tell whether the #ideas in a piece of writing came from a human or an AI? As AI writing becomes common, policies on AI use focus on who came up with the ideas in the text rather than who wrote it. AI detectors answer the latter, not the former. We release #IdeaLens, a detector of AI ideas.

View full thread

Traditional detectors are very accurate when #prose and #ideas come from the same source. But #IdeaLens is the only detector that can detect AI ideas hidden behind human prose.

On conventional AI detection benchmarks, where prose and ideas come from the same source, IdeaLens correctly flags 91.1% as AI at 0.6% FPR, outperforming all open-source detectors. ProseLens does even better, flagging 98.9% at 0.7% FPR.

Can you tell whether the #ideas in a piece of writing came from a human or an AI? As AI writing becomes common, policies on AI use focus on who came up with the ideas in the text rather than who wrote it. AI detectors answer the latter, not the former. We release #IdeaLens, a detector of AI ideas.

We represent documents as outlines of their ideas and train #IdeaLens to fit Pangram 3 labels after paraphrasing the outlines to remove stylistic cues from the source. As a control, we train #ProseLens directly on the raw documents.

Anyone going to #EMNLP2026? Please consider helping us understand how people use machine translation tools when abroad! (and if you can't, please share!)

Ehud Reiter@ehudreiter.bsky.social

Going to EMNLP 2026 in Budapest? We are looking for 20-30 EMNLP attendees to help us understand how you handle language barriers during a conference trip. Contact Yujun Wang: y.wang2.25@abdn.ac.uk @weizhaonlp.bsky.social, @markar.bsky.social, Pinzhen Chen, Yujun Wang

Marzena Karpinska reposted

Happy to see these papers accepted #Neurips2026 🎊 🎊 🎊 Please come and find us at the conference! More details & links in 🧵

Marzena Karpinska reposted

EMNLP 2026 is coming up soon and we will be there! See you in Budapest 🇭🇺 Oct 24–29 #EMNLP2026 Check out our work!

Marzena Karpinska reposted

I'm recruiting 1-2 PhD students to join our lab in Fall 2027, through either Computer Science or Information Science at the University of Colorado Boulder! Looking for people with interests in NLP plus [healthcare | literary studies | narratives | social media | etc.]. Join us! 🏔️☀️

cls-lab.comCLS Lab — Culture, Language, & Systems | University of Colorado BoulderThe Culture, Language, & Systems Lab at CU Boulder studies the language systems that transmit and shape modern culture, from internet platforms to literary archives to language models.

still high, but better than allowing for 40+ per person, not sure how (who) will the shared first authorship be checked....

ACL 2027@aclmeeting.bsky.social

Additionally, per-author quotas are introduced, where authors are capped at 20 total submissions and 5 first-author (including shared first-author) submissions per cycle. More details wil be posted on the ACL web and here shortly. #NLProc

Marzena Karpinska reposted

ACL Sustainable Reviewing Policy: We are introducing changes in the @ReviewAcl reviewing and submissions. The changes will cap authors and introduce changes in the reviewing to keep our community sustainable. #NLProc #NLP

Marzena Karpinska reposted

Regular reminder that we have an alt-ARR slack workspace where ACs and SACs can support each other through the sometimes confusing process of the ARR cycle! Post or DM me a good email address for a Slack invitation and I will add you. #EMNLP2026

this checkbox at #arr really seems like a get-out-of-jail-free card; if we really need to allow for lang edits, why not require the reviewers to submit their pre-GPTed draft along with the edited one? 😭😭😭

Marzena Karpinska reposted

Text as Data is happening at Berkeley right before COLM. Please share!

Denis@dpeskoff.bsky.social

Do you work across computational methods, social sciences, and the humanities? Submit to Text as Data 2026! 📄 One-page submissions 🔓 Non-archival ⏰ Due August 1 📍 October 5 @UCBerkeley tada2026.org

Marzena Karpinska reposted

Peer review was one of the most-discussed topics at #ACL2026 . Many folks were concerned about the incredible growth in the number of ARR submissions (17K for the May26 ARR cycle 😱), and even more shocked that ~40% didn't have any authors qualified to review. What is going on?? I did some digging...

Marzena Karpinska reposted

hot take: don’t even use it for polishing your reviews. as an AC and SAC, the last thing i care about is your grammar, spelling, or formatting. i get the presentation issues with papers, where you might face reviewers’ language biases, but you really don’t need to worry about this for reviews.

Marzena Karpinska@markar.bsky.social

I think I will be posting this after each #ARR cycle: Please 🥺🙏 let's prohibit AI review writing. Otherwise, we will get lazy reviewers claiming they wrote bullet points and used AI only to put that in prose.

I think I will be posting this after each #ARR cycle: Please 🥺🙏 let's prohibit AI review writing. Otherwise, we will get lazy reviewers claiming they wrote bullet points and used AI only to put that in prose.

Please 🙏 AI detectors are not that bad, if something sounds AI and @pangram alike say it's 100% AI it is not just using a model to polish and it can get sloppy. Even if the author meant well they can miss something model misrepresented.

5 years later MTurk is on its way out. A lot has improved in open-ended gen eval, but it still suffers from underreporting, lack of statistical analysis, and sometimes sloppy design. Perhaps authors should always do their own tasks to understand the implications of how they designed evals.

Ethan Mollick@emollick.bsky.social

Unsurprising but still big: MTurk is on its way out, killed by AI. Mechanical Turk was a mainstay of social & survey research through the 2010s, as it allowed you to quickly buy access to many representative humans. It was pretty good at it, until LLMs came along and everyone started using AI

In the spirit of #paperoverflow looking for emergency reviewers: - aspect-based summarization - hallucination detection - evaluation invariance - evaluation, evaluation bias - fine-grained evaluation for some, the reviewers filled in delay but became unresponsive later HELP, this house is on fire 🔥

Marzena Karpinska reposted

For the last few months, I've been working with my new colleague at SFU @markar.bsky.social, students @yvesfrtl.bsky.social, @adampodoxin.bsky.social, Ty Brassington, and Roman Grundkiewicz of Microsoft, trying to understand the quality of machine translation for literary texts. ...

yvesfrtl.bsky.social@yvesfrtl.bsky.social

📚 AI-written stories get attention, but AI-translated literature is quietly shaping how readers experience the author So what gets lost ⁉️ 🤖 AI translation into English is readable and often ‘fine’ ✍️ But human translation is valued more ‼️Both vary in quality BUT AI more, even within a single book

Marzena Karpinska reposted

It's invaluable in good literature to have someone who has actual, lived experience and a deep cultural and historical understanding of the literature performing the translation @jricole.bsky.social translation of the Rubaiyat is a great example. Omar's words ring out with truth and subtle humor.

Marzena Karpinska@markar.bsky.social

A lot of people focus on AI-written fiction, but what about AI literary translation? 📚 We find that AI translation can be readable.. ‼️BUT it also flattens characters' voices 🧙and is less immersive 🫣 than published human translations. Below is my favorite quote from a reader 📖 lait.cs.sfu.ca

A lot of people focus on AI-written fiction, but what about AI literary translation? 📚 We find that AI translation can be readable.. ‼️BUT it also flattens characters' voices 🧙and is less immersive 🫣 than published human translations. Below is my favorite quote from a reader 📖 lait.cs.sfu.ca

yvesfrtl.bsky.social@yvesfrtl.bsky.social

📚 AI-written stories get attention, but AI-translated literature is quietly shaping how readers experience the author So what gets lost ⁉️ 🤖 AI translation into English is readable and often ‘fine’ ✍️ But human translation is valued more ‼️Both vary in quality BUT AI more, even within a single book

Marzena Karpinska reposted

📚 AI-written stories get attention, but AI-translated literature is quietly shaping how readers experience the author So what gets lost ⁉️ 🤖 AI translation into English is readable and often ‘fine’ ✍️ But human translation is valued more ‼️Both vary in quality BUT AI more, even within a single book

One more thing @tuhinchakr.bsky.social 's post reminded me of... people tend to rationalize and see things not there. We saw it already in GPT-2 stories - we *expect* things to *mean* something, so we tend to see things that are not there... (link to this old paper: aclanthology.org/2021.emnlp-m...)

Marzena Karpinska@markar.bsky.social

this is how massive illusion of 'creativity' gets crushed... please read it to understand why models may appear to produce coherent text but are in fact Frankenstein factory ...

this is how massive illusion of 'creativity' gets crushed... please read it to understand why models may appear to produce coherent text but are in fact Frankenstein factory ...

Tuhin Chakrabarty@tuhinchakr.bsky.social

Ran some 🧪 to 🔬 why the Granta short story was certainly 🤖 generated. A lot of bad writing happens because AI hasn’t learned aesthetics. It has simply memorized the whole internet and called it a day. So sure, maybe you don't trust AI detectors. But you can trust your own eyes. #AISlop

Marzena Karpinska reposted

"Load bearing," "I keep coming back to," "Not just X, but Y" A curse of using AI a lot is that you realize how much of the writing around you is just AI, now People who don't use AI have historically been unable to identify AI prose on sight, but those who use it a lot can spot the tells easily