20 results for “community metrics”
CS papers onlyHybrid search: Keyword + semantic, ranked by combined score.ⓘ
Want pure semantic search? Try claim verification →
Tianjiao Li, Kai Zhao, Xiang Li, Yang Liu +1 more
The paper introduces CASTER, a new human-centric task for evaluating User-Generated Content (UGC) resonance, and proposes MEDEA, an architecture that uses a Social Chain-of-Thought mechanism to simula…
Pushwitha Krishnappa, Amit Das, Vinija Jain, Aman Chadha +1 more
This paper evaluates the performance of language models on open-ended question answering tasks using a new dataset and finds that current metrics are not effective in both validity and discriminative…
Jan Batzner, Sree Harsha Nelaturu, Anastassia Kornilova, Jon Crall +43 more
The paper introduces Every Eval Ever, a shared schema and community-crowdsourced repository for AI evaluation results to standardize and compare AI evaluations.
The paper introduces TeleHunt, a comprehensive framework and tool that systematically evaluates various strategies for efficiently discovering cybercriminal communities operating on Telegram.
This paper introduces a new metric to measure organizational cohesion in microservice ecosystems and analyzes its evolution in several open-source systems.
This paper presents Entangle, a data-driven analysis of the open-source quantum computing ecosystem on GitHub.
This study provides an ecosystem-scale measurement of commit signing on GitHub, finding that current signing adoption rates are misleading and that developers struggle to maintain consistent, long-ter…
Roy Ricaldi, Maximilian Schafer, Philipp Zech, Luca Allodi +2 more
This study provides a longitudinal analysis of dark web content, revealing that cybercrime discussions are dominated by a few persistent core topics rather than rapidly shifting themes.
Taein Kim, David Jiang, Yuepeng Hu, Yuqi Jia +1 more
The paper presents a large-scale study demonstrating that tool cloning is a pervasive and severe source of hidden duplication in agent-tool ecosystems, necessitating changes in how tool diversity is m…
Shreeyans Arora, Utsav Avaiya, Zera Lyngkhoi, Hem Gosalia +3 more
The paper presents AuditKIT, a library for evaluating language models and agents by recording various measurements in one result representation.
The paper introduces WebKnoGraph, an open-source framework for systematically evaluating internal linking strategies on websites by modeling the site as a graph and assessing trade-offs between author…
Yugao Zhu, Zhicheng Guo, Yuchao Wu, Mengming Li +2 more
This paper proposes a new framework called ComPart for hypergraph partitioning that integrates community detection methods to improve solution quality.
The paper introduces CRAFT, a method for converting evaluation datasets into model-specific diagnoses of weak capabilities, achieving stronger results than baselines on four open source models and two…
This paper investigates unequal exposure of identical texts in custom feeds on Bluesky platform.
The Internet Quality Barometer (IQB) framework maps Internet measurement data to actionable insights about Internet quality.
This study analyzed I2P's routing topology and found no significant evidence that peer selection is influenced by geographic location, suggesting highly random global mixing.