Intelligent systems often depend on data provided by information agents, for example, sensor data or crowdsourced human computation. Providing accurate and relevant data requires costly effort that agents may not always be willing to provide. Thus, it becomes important not only to verify the correctness of data, but also to provide incentives so that agents that provide high-quality data are rewarded while those that do not are discouraged by low rewards.

We cover different settings and the assumptions they admit, including sensing, human computation, peer grading, reviews, and predictions. We survey different incentive mechanisms, including proper scoring rules, prediction markets and peer prediction, Bayesian Truth Serum, Peer Truth Serum, Correlated Agreement, and the settings where each of them would be suitable. As an alternative, we also consider reputation mechanisms. We complement the game-theoretic analysis with practical examples of applications in prediction platforms, community sensing, and peer grading.

Autorentext

Boi Faltings is a full professor at Ecole Polytechnique Federale de Lausanne (EPFL) and has worked in AI since 1983. He is one of the pioneers on the topic of mechanisms for truthful information elicitation, with the first work dating back to 2003. He has taught AI and multi-agent systems to students at EPFL for 28 years. He is a fellow of AAAI and ECCAI and has served on program committee and editorial boards of the major conferences and journals in Artificial Intelligence.Goran Radanovic has been a post-doctoral fellow at Harvard University since 2016. He received his Ph.D. from the Swiss Federal Institute of Technology and has worked on the topic of mechanisms for information elicitation since 2011. His work has been published mainly at AI conferences.