A Causal Framework for Mining Helpfulness Voting: Learning Information Quality and Community Dynamics

Efficient access to high-quality information is crucial for online platforms that rely on helpfulness votes to evaluate and rank user-generated content. These votes are designed to reflect collective judgments about information quality. However, the voting process is often affected by position bias (where prominently displayed items receive more attention) and herding bias (where users are influenced by previous votes). These biases can distort evaluations, leading to overestimating popular content rather than the most valuable information. This has sparked interest in understanding how these biases interact and reinforce each other and developing more accurate and fairer methods for assessing information quality. In essay one “From Popularity to Meritocracy: Information Monopoly and Evolution of Excellence in Online Communities”, I propose the Evolution of Excellence (EoE), a theoretical framework that explains the position and herding biases along with their monopolistic reinforcement. To quantify these biases, we map the diverse voting behaviors across 120 major StackExchange communities onto two behavioral axes, further clustering the communities into four distinct information categories. We also demonstrate the behavioral evolution of individual communities over the years, highlighting longitudinal insights into the underlying social process. Our findings can help service providers secure community-specific interventions that foster meritocracy by promoting valuable yet under-recognized information without running expensive A/B tests or model training. In essay two “Finding Information Quality: Counterfactual Voting Adjustment for Quality Assessment and Voting Fairness in Online Platforms with Helpfulness Evaluation”, for fairer assessment of information quality, we propose the Counterfactual Voting Adjustment (CVA), a causal framework that account for the context in which individual votes are cast. Through preliminary and semi-synthetic experiments, we show that CVA effectively models the position and herding biases, accurately recovering the predefined content quality. In real experiment, we demonstrate that reranking content based on the learned quality by CVA exhibits stronger alignment with both user sentiment and quality evaluation assessed by GPT-4o, outperforming system rankings based on aggregated votes and model-based rerankings without causal inference. Beyond the individual quality inference, our embeddings offer comparative insights into the behavioral dynamics of expert user groups across major StackExchange communities.