Publications

1. A Simple, Statistically Robust Test of Discrimination. With Johann Gaebler. Working paper. [ Data & Code ]
2. Automated Reminders Reduce Incarceration for Missed Court Dates: Evidence from a Text Message Experiment. With Alex Chohlas-Wood, Madison Coots, Joe Nudell, Julian Nyarko, Emma Brunskill, and Todd Rogers. Working paper.
3. Mitigating Included- and Omitted-Variable Bias in Estimates of Disparate Impact. With Jongbin Jung, Sam Corbett-Davies, Johann Gaebler, and Ravi Shroff. Working paper.
4. Auditing the Use of Language Models to Guide Hiring Decisions. With Johann Gaebler, Aziz Huq, and Prasanna Tambe. Behavioral Science & Policy (forthcoming).
5. Racial Bias in Clinical and Population Health Algorithms: A Critical Review of Current Debates. With Madison Coots, Kristin A. Linn, Amol S. Navathe, and Ravi B. Parikh. Annual Review of Public Health (forthcoming).
6. Learning to Be Fair: A Consequentialist Approach to Equitable Decision-Making. With Alex Chohlas-Wood, Madison Coots, Henry Zhu, and Emma Brunskill. Management Science (forthcoming). [ Code ]
7. A Framework for Considering the Value of Race and Ethnicity in Estimating Disease Risk. With Madison Coots, Soroush Saghafian, and David Kent. Annals of Internal Medicine, 2024. [ Code ]
8. Guidance for Unbiased Predictive Information for Healthcare Decision-Making and Equity (Guide): Considerations When Race May Be a Prognostic Factor. With Keren Ladin, John Cuddeback, O. Kenrik Duru, William Harvey, Jinny G. Park, Jessica K. Paulus, Joyce Sackey, Richard Sharp, Ewout Steyerberg, Berk Ustun, David van Klaveren, Saul N. Weingart, and David M. Kent. npj Digital Medicine, Vol. 7, 2024.
9. Reconciling Legal and Empirical Conceptions of Disparate Impact: An Analysis of Police Stops Across California. With Joshua Grossman and Julian Nyarko. Journal of Law and Empirical Analysis, 2024.
10. Risk Scores, Label Bias, and Everything but the Kitchen Sink. With Michael Zanger-Tishler and Julian Nyarko. Science Advances, Vol. 10, 2024. [ Appendix - Code ]
11. The Disparate Impacts of College Admissions Policies on Asian American Applicants. With Joshua Grossman, Sabina Tomkins, and Lindsay Page. Scientific Reports, Vol. 14, 2024. [ Essay in the Boston Globe - Appendix - Data & Code ]
12. Empirical Approaches to Identify Systemic Discrimination in Policing. With Alex Chohlas-Wood, Marissa Gerchick, Aziz Huq, Amy Shoemaker, Ravi Shroff, and Keniel Yao. Inequality Reader (forthcoming).
13. LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models. With Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N. Rockmore, and Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, Jessica Wu, Joe Nudell, Joel Niklaus, John Nay, Jonathan H. Choi, Kevin Tobia, Margaret Hagan, Megan Ma, Michael Livermore, Nikon Rasumov-Rahe, Nils Holzenberger, Noam Kolt, Peter Henderson, Sean Rehaag, Shang Gao, Spencer Williams, Sunny Gandhi, Tom Zur, Varun Iyer, and Zehua Li. Advances in Neural Information Processing Systems (NeurIPS 2023). [ Project site ]
14. Showing High-Achieving College Applicants Past Admissions Outcomes Increases Undermatching. With Sabina Tomkins, Joshua Grossman, and Lindsay Page. Proceedings of the National Academy of Sciences, Vol. 120, 2023. [ Data & Code ]
15. Forgotten but Not Gone: A Multi-State Analysis of Modern-Day Debt Imprisonment. With Johann Gaebler, Phoebe Barghouty, Sarah Vicol, and Cheryl Phillips. PLOS One, Vol. 18, 2023. [ Data & Code ]
16. The Measure and Mismeasure of Fairness. With Sam Corbett-Davies, Johann Gaebler, Hamed Nilforoshan, and Ravi Shroff. Journal of Machine Learning Research, Vol. 24, 2023. [ Discussion on Moral Science Podcast - Data & Code ]
17. Designing Equitable Algorithms. With Alex Chohlas-Wood, Madison Coots, and Julian Nyarko. Nature Computational Science, Vol. 3, 2023. [ Code ]
18. Popular Support for Balancing Equity and Efficiency in Resource Allocation: A Case Study in Online Advertising to Increase Welfare Program Awareness. With Allison Koenecke, Eric Giannella, and Robb Willer. The 17th International Conference On Web and Social Media (ICWSM 2023).
19. Racial Bias as a Multi-Stage, Multi-Actor Problem: An Analysis of Pretrial Detention. With Joshua Grossman and Julian Nyarko. Journal of Empirical Legal Studies, Vol. 20, 2023.
20. Blocks as Geographic Discontinuities: The Effect of Polling Place Assignment on Voting. With Sabina Tomkins, Keniel Yao, Johann Gaebler, Tobias Konitzer, David Rothschild, and Marc Meredith. Political Analysis, 2022. [ Appendix - Data ]
21. Measuring Racial and Ethnic Disparities in Traffic Enforcement with Large-Scale Telematics Data. With William Cai, Johann Gaebler, Justin Kaashoek, Lisa Pinals, and Samuel Madden. PNAS Nexus, Vol. 1, 2022. [ Essay in The Washington Post - Data & Code ]
22. Causal Conceptions of Fairness and their Consequences. With Hamed Nilforoshan, Johann Gaebler, and Ravi Shroff. International Conference on Machine Learning (ICML 2022). [ Received an Outstanding Paper Award at ICML 2022 ]
23. Adaptive Sampling Strategies to Construct Equitable Training Datasets. With William Cai, Ro Encarnacion, Bobbie Chern, Sam Corbett-Davies, Miranda Bogen, and Stevie Bergman. Conference on Fairness, Accountability, and Transparency (FAccT 2022).
24. Identifying and Measuring Excessive and Discriminatory Policing. With Alex Chohlas-Wood, Marissa Gerchick, Aziz Huq, Amy Shoemaker, Ravi Shroff, and Keniel Yao. University of Chicago Law Review, Vol. 89, 2022.
25. A Causal Framework for Observational Studies of Discrimination. With Johann Gaebler, William Cai, Guillaume Basse, Ravi Shroff, and Jennifer Hill. Statistics and Public Policy, Vol. 9, 2022. [ Code ]
26. Probability Paths and the Structure of Predictions over Time. With Zhiyuan (Jerry) Lin and Hao Sheng. Advances in Neural Information Processing Systems (NeurIPS 2021).
27. Breaking Taboos in Fair Machine Learning: An Experimental Study. With Julian Nyarko and Roseanna Sommers. Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO 2021). [ Essay in the Boston Globe ]
28. The Accuracy, Equity, and Jurisprudence of Criminal Risk Assessment. With Ravi Shroff, Jennifer Skeem, and Christopher Slobogin. Research Handbook on Big Data Law, 2021.
29. Bandit Algorithms to Personalize Educational Chatbots. With William Cai, Joshua Grossman, Zhiyuan (Jerry) Lin, Hao Sheng, Johnny Tian-Zheng Wei, and Joseph Jay Williams. Machine Learning, Vol. 110, 2021.
30. Surveilling Surveillance: Estimating the Prevalence of Surveillance Cameras with Street View Data. With Hao Sheng and Keniel Yao. Conference on AI, Ethics, and Society (AIES 2021). [ View camera locations - Data & Code ]
31. Blind Justice: Algorithmically Masking Race in Charging Decisions. With Alex Chohlas-Wood, Joe Nudell, Keniel Yao, Zhiyuan (Jerry) Lin, and Julian Nyarko. Conference on AI, Ethics, and Society (AIES 2021).
32. Simple Rules to Guide Expert Classifications. With Jongbin Jung, Connor Concannon, Ravi Shroff, and Daniel G. Goldstein. Journal of the Royal Statistical Society: Series A, Vol. 183, 2020. [ Essay in Harvard Business Review ]
33. A Large-scale Analysis of Racial Disparities in Police Stops Across the United States. With Emma Pierson, Camelia Simoiu, Jan Overgoor, Sam Corbett-Davies, Daniel Jenson, Amy Shoemaker, Vignesh Ramachandran, Phoebe Barghouty, Cheryl Phillips, and Ravi Shroff. Nature Human Behaviour, Vol. 4, 2020. [ Stanford Open Policing Project - Supporting Information - Essay in Slate ]
34. Racial Disparities in Automated Speech Recognition. With Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John Rickford, and Dan Jurafsky. Proceedings of the National Academy of Sciences, Vol. 117, 2020. [ Listen to audio samples - Data & Code ]
35. The Limits of Human Predictions of Recidivism. With Zhiyuan (Jerry) Lin, Jongbin Jung, and Jennifer Skeem. Science Advances, Vol. 6, 2020. [ Essay in The Washington Post - Data & Code ]
36. One Person, One Vote: Estimating the Prevalence of Double Voting in U.S. Presidential Elections. With M. Meredith, M. Morse, D. Rothschild, and H. Shirani-Mehr. American Political Science Review, Vol. 114, 2020. [ Essay in Slate - Interview on This American Life ]
37. Fair Allocation through Selective Information Acquisition. With William Cai, Johann Gaebler, and Nikhil Garg. Conference on AI, Ethics, and Society (AIES 2020).
38. Bayesian Sensitivity Analysis for Offline Policy Evaluation. With Jongbin Jung, Ravi Shroff, and Avi Feller. Conference on AI, Ethics, and Society (AIES 2020).
39. Partisan Selective Exposure in Online News Consumption: Evidence from the 2016 Presidential Campaign. With Erik Peterson and Shanto Iyengar. Political Science Research and Methods, 2020.
40. An Experimental Study of Structural Diversity in Social Networks. With Jessica Su, Krishna Kamath, Aneesh Sharma, and Johan Ugander. The 14th International Conference On Web and Social Media (ICWSM 2020). [ Awarded Best Paper at ICWSM 2020 ]
41. Studying the “Wisdom of Crowds” at Scale. With Camelia Simoiu, Chiraag Sumanth, and Alok Mysore. The 7th AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2019). [ Awarded Best Paper at HCOMP 2019 ]
42. “I was told to buy a software or lose my computer. I ignored it”: A study of ransomware. With Camelia Simoiu, Christopher Gates, and Joseph Bonneau. Fifteenth Symposium on Usable Privacy and Security (SOUPS 2019).
43. Guiding Prosecutorial Decisions with an Interpretable Statistical Model. With Zhiyuan (Jerry) Lin and Alex Chohlas-Wood. Conference on AI, Ethics, and Society (AIES 2019).
44. Machine Learning, Health Disparities, and Causal Reasoning. With Steven Goodman and Mark Cullen. Annals of Internal Medicine, Vol. 169, 2018.
45. Disentangling Bias and Variance in Election Polls. With Houshmand Shirani-Mehr, David Rothschild, and Andrew Gelman. Journal of the American Statistical Association, Vol. 113, 2018. [ Essay in The New York Times ]
46. Fast Threshold Tests for Detecting Discrimination. With Emma Pierson and Sam Corbett-Davies. The 21st International Conference on Artificial Intelligence and Statistics (AISTATS 2018). [ Appendix; Awarded Best Paper at AISTATS 2018 ]
47. Creating Crowdsourced Research Talks at Scale. With Rajan Vaish, Shirish Goyal, and Amin Saberi. Proceedings of the 27th International World Wide Web Conference (WWW 2018). [ Video clip - Stanford Scholar ]
48. Online, Opt-in Surveys: Fast and Cheap, but are they Accurate?. With Adam Obeng and David Rothschild. Working paper.
49. Crowd Research: Open and Scalable University Laboratories. With Rajan Vaish, Michael S. Bernstein, et al. Proceedings of the 30th Annual Symposium on User Interface Software and Technology (UIST 2017).
50. Algorithmic Decision Making and the Cost of Fairness. With Sam Corbett-Davies, Emma Pierson, Avi Feller, and Aziz Huq. Proceedings of the 23rd Conference on Knowledge Discovery and Data Mining (KDD 2017). [ Essay in New York Times - Essay in Washington Post - Tutorial on fair ML ]
51. The Problem of Infra-marginality in Outcome Tests for Discrimination. With Camelia Simoiu and Sam Corbett-Davies. Annals of Applied Statistics, Vol. 11, 2017. [ Data - code ]
52. De-Anonymizing Web Browsing Data with Social Networks. With Jessica Su, Ansh Shukla, and Arvind Narayanan. Proceedings of the 26th International World Wide Web Conference (WWW 2017). [ Essay in Slate ]
53. Combatting Police Discrimination in the Age of Big Data. With Maya Perelman, Ravi Shroff, and David Sklansky. New Criminal Law Review, Vol. 20, 2017. [ Essay in The Huffington Post ]
54. Understanding Emerging Threats to Online Advertising. With Ceren Budak, Justin Rao, and Georgios Zervas. Proceedings of the 17th ACM Conference on Economics & Computation (EC 2016).
55. Personalized Risk Assessments in the Criminal Justice System. With Justin Rao and Ravi Shroff. The American Economic Review: Papers and Proceedings, Vol. 106, 2016.
56. High-Frequency Polling with Non-Representative Data. With Andrew Gelman, David Rothschild, and Wei Wang. Routledge Studies in Global Information, Politics and Society, 2016.
57. The Mythical Swing Voter. With David Rothschild, Andrew Gelman, and Doug Rivers. Quarterly Journal of Political Science, Vol. 11, 2016.
58. Filter Bubbles, Echo Chambers, and Online News Consumption. With Seth Flaxman and Justin Rao. Public Opinion Quarterly, Vol. 80, 2016. [ Supporting Information ]
59. Fair and Balanced? Quantifying Media Bias through Crowdsourced Content Analysis. With Ceren Budak and Justin Rao. Public Opinion Quarterly, Vol. 80, 2016. [ Supporting Information - Data ]
60. Precinct or Prejudice? Understanding Racial Disparities in New York City's Stop-and-Frisk Policy. With Justin Rao and Ravi Shroff. Annals of Applied Statistics, Vol. 10, 2016. [ Processed stop-and-frisk data as an RData file; original NYPD data ]
61. The Effect of Recommendations on Network Structure. With Jessica Su and Aneesh Sharma. Proceedings of the 25th International World Wide Web Conference (WWW 2016).
62. The Structural Virality of Online Diffusion. With Ashton Anderson, Jake Hofman, and Duncan J. Watts. Management Science, Vol. 62, 2016.
63. Forecasting Elections with Non-Representative Polls. With Wei Wang, David Rothschild, and Andrew Gelman. International Journal of Forecasting, Vol 31, 2015.
64. Political Ideology and Racial Preferences in Online Dating. With Ashton Anderson, Gregory Huber, Neil Malhotra, and Duncan J. Watts. Sociological Science, Vol. 1, 2014. [  Rejoinder to a comment on our paper ]
65. Predicting Individual Behavior with Social Networks. With Daniel G. Goldstein. Marketing Science, Vol. 33, 2014.
66. Sharding Social Networks. With Quang Duong, Jake Hofman, and Sergei Vassilvitskii. Proceedings of the Fifth Conference on Web Search and Data Mining (WSDM 2012).
67. Respondent Driven Sampling—Where We Are and Where Should We be Going?. With Richard White, Amy Lansky, David Wilson, Wolfgang Hladik, Avi Hakim and Simon DW Frost. Sexually Transmitted Infections, Vol. 88, No. 6, 2012, 397-399. [ Supporting Information ]
68. The Structure of Online Diffusion Networks. With Duncan J. Watts and Daniel G. Goldstein. Proceedings of the 13th ACM Conference on Economics & Computation (EC 2012).
69. Who Does What on the Web: Studying Web Browsing Behavior at Scale. With Jake Hofman and M. Irmak Sirer. Proceedings of the 6th International Conference on Weblogs and Social Media (ICWSM 2012).
70. Predicting Consumer Behavior with Web Search. With Jake Hofman, Sébastien Lahaie, David Pennock, and Duncan Watts. Proceedings of the National Academy of Sciences, Vol 107, No. 41, 2010, 17486-17490.
71. Real and Perceived Attitude Agreement in Social Networks. With Winter Mason and Duncan Watts. Journal of Personality and Social Psychology, Vol. 99, No. 4, 2010, 611-621.
72. Assessing Respondent-Driven Sampling. With Matthew Salganik. Proceedings of the National Academy of Sciences, Vol. 107, No. 15, 2010, 6743-6747. [ Supporting Information - Project 90 Data ]
73. Prediction Without Markets. With Daniel Reeves, Duncan Watts, and David Pennock. Proceedings of the 11th ACM Conference on Economics & Computation (EC 2010).
74. Anatomy of the Long Tail: Ordinary People With Extraordinary Tastes. With Andrei Broder, Evgeniy Gabrilovich, and Bo Pang. Proceedings of the Third Conference on Web Search and Data Mining (WSDM 2010).
75. Contract Auctions for Sponsored Search. With Sébastien Lahaie and Sergei Vassilvitskii. Proceedings of the 5th Workshop on Internet and Network Economics (WINE 2009).
76. Collective Revelation: A Mechanism for Self-Verified, Weighted, and Truthful Predictions. With Daniel Reeves and David Pennock. Proceedings of the 10th ACM Conference on Economics & Computation (EC 2009).
77. CentMail: Rate Limiting via Certified Micro-Donations. With Jake Hofman, John Langford, David Pennock, and Daniel Reeves. Proceedings of the 6th Conference on Email and Anti-Spam (CEAS 2009). [ Short version at WWW 2009, Developer's Track ]
78. Respondent-Driven Sampling as Markov Chain Monte Carlo. With Matthew Salganik. Statistics in Medicine, Vol. 28, No. 17, 2009, 2202-2229.
79. Social Search in “Small-World” Experiments. With Roby Muhamad and Duncan Watts. Proceedings of the 18th International World Wide Web Conference (WWW 2009).
80. Predictive Indexing for Fast Search. With John Langford and Alex Strehl. Advances in Neural Information Processing Systems (NIPS 2008).
81. Yoopick: A Combinatorial Sports Prediction Market. With David Pennock, Daniel Reeves, and Cong Yu. Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI 2008).
82. Pricing Combinatorial Markets for Tournaments. With Yiling Chen and David Pennock. Proceedings of the 40th ACM Symposium on Theory of Computing (STOC 2008).
83. Horseshoes in Multidimensional Scaling and Local Kernel Methods. With Persi Diaconis and Susan Holmes. Annals of Applied Statistics, Vol. 2, No. 3, 2008, 777-807.
84. An Invisible Minority: Asian-Americans in Mathematics. Notices of the American Mathematical Society, Vol. 53, No. 8, 2006, 878-882.
85. Analysis of Top to Bottom-k Shuffles. Annals of Applied Probability, Vol. 16, No. 1, 2006, 30-55.
86. Mixing Time Bounds via the Spectral Profile. With Ravi Montenegro and Prasad Tetali. Electronic Journal of Probability, Vol. 11, 2006, 1-26.
87. Eluding Carnivores: File Sharing with Strong Anonymity. With Emin Gün Sirer, Mark Robson, and Doğan Engin. Proceedings of the 11th ACM SIGOPS European Workshop. 2004.
88. Modified Logarithmic Sobolev Inequalities for Some Models of Random Walk. Stochastic Processes and Their Applications, Vol. 114, 2004, 51-79.