Webis Group

Webis Group

@webis.de

Information is nothing without retrieval The Webis Group contributes to information retrieval, natural language processing, machine learning, and symbolic AI.

662 followers698 following270 posts

Webis Group
@webis.de

Call for Participation: Deadlines extended for UniAgent, the 1st Shared Task on Agentic AI in University Administration at the CIKM 2026 AnalytiCup in Rome. Registration until Oct 3, submission Oct 20. Real administrative cases; we host the LLMs. Details: uniagent.webis.de/cikm26/uniag...

uniagent.webis.deUniAgent at CIKM 2026 - Agentic AI in University AdministrationA shared task on agentic AI in university administration, part of the CIKM 2026 AnalytiCup: build agents that solve standardized administrative tasks under controlled tool access and expert-judged ref...
Webis Group
@webis.de

Call for Participation: UniAgent, the 1st Shared Task on Agentic AI in University Administration, at the CIKM 2026 AnalytiCup in Rome. Registration closes Sep 30. Submission deadline Oct 23. Details: uniagent.webis.de/cikm26/uniag...

uniagent.webis.deUniAgent at CIKM 2026 - Agentic AI in University AdministrationA shared task on agentic AI in university administration, part of the CIKM 2026 AnalytiCup: build agents that solve standardized administrative tasks under controlled tool access and expert-judged ref...
Webis Group
@webis.de

The data spans 7 text domains: 🌐 Web: Wikipedia, GitHub, social media 💬 Political: Parliamentary proceedings, speeches ⚖️ Legal: Court decisions, federal & EU law 📰 News: Newspaper archives 🏦 Economics: public tenders 📚 Cultural: Digital heritage collections 🔬 Scientific: Papers, books, journals

Webis Group
@webis.de

The current problem: training data is primarily sourced from Web crawls, which give you scale but unclear licensing. This blocks models from commercial deployment and research. We took a different path: systematically collecting German text from 41 institutional sources with explicit open licenses.

Webis Group
@webis.de

This means: ✅ Every document has verifiable usage rights (min. CC-BY-SA 4.0 and allows commercial use) ✅ Full institutional provenance for reduced compliance risks ✅ Systematic PII removal + quality filtering, ready for training ✅ Rich metadata for downstream customization

Webis Group
@webis.de

Thrilled to announce that Matti Wiegmann has successfully defended his PhD! 🎉🧑‍🎓 Huge congratulations on this incredible achievement! #PhDDefense #AcademicMilestone

Webis Group
@webis.de

Happy to share that our paper "The Viability of Crowdsourcing for RAG Evaluation" received the Best Paper Honourable Mention at #SIGIR2025! Very grateful to the community for recognizing our work on improving RAG evaluation.  📄 webis.de/publications...

Webis Group
@webis.de

Our paper on self-distillation for training bi-encoders got accepted at #ICTIR2025! By exploiting pretrained encoder capabilities, our approach eliminates expensive teacher models and batch sampling while maintaining the same effectiveness.

View full thread
Webis Group
@webis.de

Results on BEIR demonstrate that our method matches teacher distillation effectiveness, while using only 13.5% of the data and achieving 3-15x training speedup. This makes effective bi-encoder training more accessible, especially for low-resource settings.

Webis Group
@webis.de

Congratulations to the authors @lgnp.bsky.social @deckersniklas.bsky.social @martin-potthast.com @hscells.bsky.social ! 📄 Preprint: arxiv.org/abs/2407.21515 💻 Code: github.com/webis-de/ada...

arxiv.orgLearning Effective Representations for Retrieval Using Self-Distillation with Adaptive Relevance MarginsRepresentation-based retrieval models, so-called biencoders, estimate the relevance of a document to a query by calculating the similarity of their respective embeddings. Current state-of-the-art bien...
Webis Group
@webis.de

Our paper on self-distillation for training bi-encoders got accepted at #ICTIR2025! By exploiting pretrained encoder capabilities, our approach eliminates expensive teacher models and batch sampling while maintaining the same effectiveness.

Webis Group
@webis.de

The key idea: we can use the similarity predicted by the encoder itself between positive and negative documents to scale a traditional margin loss. This performs implicit hard negative mining and is hyperparameter-free.

Webis Group
@webis.de

Our paper titled “The Two Paradigms of LLM Detection: Authorship Attribution vs. Authorship Verification” has been accepted to #ACL2025 (Findings). downloads.webis.de/publications... We discuss why LLM detection is a one-class problem and how that affects the prospective… 1/3 #ACL #NLP #ARR #LLM

Webis Group
@webis.de

…limits of the field. We argue that as LLMs improve, detection will not necessarily become impossible, but it will be limited by the capabilities and theoretical boundaries of the field of authorship verification. We conduct a series of exploratory analyses to show how LLM texts differ from… 2/3

Webis Group
@webis.de

…human texts today, contextualize the findings in terms of our theoretical contribution, and use them to make an assessment of the quality and adequacy of existing LLM detection benchmarks, which tend to be constructed with authorship attribution in mind, rather than authorship verification. 3/3

Webis Group
@webis.de

Can LLM-generated ads be blocked? With OpenAI adding shopping options to ChatGPT, this question gains further importance. If you are interested in contributing to the research on LLM-based advertising, please check out our shared task: touche.webis.de/clef25/touch... More details below.

View full thread
Webis Group
@webis.de

🧵 3/4 In a lot of cases, survey participants did not notice brand or product placements in the responses. As a first step towards ad-blockers for LLMs, we created a dataset of responses with and without ads and trained classifiers on the task of identifying the ads. dl.acm.org/doi/10.1145/...

Webis Group
@webis.de

🧵 4/4 The shared task continues the research on LLM-based advertising. Participants can submit systems for two sub-tasks: First, generate responses with and without ads. Second, classify whether a response contains an ad. Submissions are open until May 10th and we look forward to your contributions.

Webis Group
@webis.de

Can LLM-generated ads be blocked? With OpenAI adding shopping options to ChatGPT, this question gains further importance. If you are interested in contributing to the research on LLM-based advertising, please check out our shared task: touche.webis.de/clef25/touch... More details below.

Webis Group
@webis.de

🧵 2/4 Given the high operating costs of LLMs, they require a business model to sustain them and advertising is a natural candidate. Hence, we have analyzed how well LLMs can blend product placements with "organic" responses and whether users are able to identify the ads. dl.acm.org/doi/10.1145/...

Webis Group
@webis.de

📢 Our paper "The Viability of Crowdsourcing for RAG Evaluation" has been accepted to #SIGIR2025 ! We compared how good humans and LLMs are at writing and judging RAG responses, assembling 1800+ responses across 3 styles, and 47K+ pairwise judgments in 7 quality dimensions. 🧵➡️

View full thread
Webis Group
@webis.de

📢 Our paper "The Viability of Crowdsourcing for RAG Evaluation" has been accepted to #SIGIR2025 ! We compared how good humans and LLMs are at writing and judging RAG responses, assembling 1800+ responses across 3 styles, and 47K+ pairwise judgments in 7 quality dimensions. 🧵➡️

Webis Group
@webis.de

🧵 2/4 Key findings: 1️⃣ Humans write best? No! LLM responses are rated better than human. 2️⃣ Essay answers? No! Bullet lists are often preferred. 3️⃣ Evaluate with BLEU? No! Reference-based metrics don't align with human preferences. 4️⃣ LLMs as judges? No! Prompted models produce inconsistent labels.

View full thread
Webis Group
@webis.de

Important Dates ---------------------- now Training Data Released May 23, 2025 Software submission May 30, 2025 Participant paper submission June 27, 2025 Peer review notification July 07, 2025 Camera-ready participant papers submission Sep 09-12, 2025 Conference

View full thread
Webis Group
@webis.de

Today we will present our poster on Query Variation Robustness of Transformer Models at #EMNLP2024. You can find us at the Information Retrieval and Text Mining 3 poster session at #EMNLP2024.

Webis Group
@webis.de

We find that simply scaling up the transformer architecture still leads to significant effectiveness drops in face of typos, keywords, ordering, and paraphrasing. We further highlight the need for more elaborate query variation datasets, which should retain the queries' semantics.

Webis Group
@webis.de

In our experiments, LLMs struggle with the task in a zero-shot setting, especially due to low precision values. Sentence transformers, however, can be finetuned to successfully detect the inserted ads and achieve precision and recall values of above 0.9 for unseen meta topics. https://t.co/VuuaW...

Webis Group
@webis.de

The Webis Generated Native Ads 2024 is the first public dataset to evaluate models on the task of detecting ads in responses of conversational search engines. It was created by simulating an advertising service for queries from popular meta topics (product/service categories). https://t.co/pjHr...

Webis Group
@webis.de

What if conversational search will be financed by inserting ads directly into generated responses? We present our work on detecting these generated native ads at #TheWebConf24. Come visit us at the short paper poster session on Thursday in the Central Ballroom. https://t.co/NRKbal57WO