Proportional Representation in Artificial Intelligence: Clustering, Alignment, and Beyond

Document Type : Review article

Authors

1 Corresponding Author, Assistant Prof, Department of Mathematics, Ardabil Branch, Islamic Azad University, Ardabil, Iran. Email: mehdi.fazli.s@gmail.com

2 Assistant Prof, Department of Accounting, Islamic Azad University, Ardabil Branch, Ardabil, Iran. Email: saeed.alipour@iau.ac.ir

10.22091/jdaid.2026.15981.1051

Abstract

Proportional representation is a foundational concept in social choice theory, seeking to ensure that the preferences of distinct groups are reflected fairly in collective decisions. As algorithmic systems increasingly shape high-stakes decisions in society, there is a growing need for principled methods that enable artificial intelligence (AI) to account for heterogeneous human values and preferences. This article explores how proportional representation can be extended beyond its classical role in voting and elections to address key challenges in modern AI. We focus on two central domains. First, we examine clustering, when data points naturally represent individuals or agents with diverse characteristics or preferences. We review recent advances that reinterpret clustering as a representation problem, introduce formal notions as a representation problem for both centroid-based and non-centroid-based clustering, and highlight algorithmic guarantees ensuring that large, cohesive groups receive influence proportional to their size. Second, we consider AI alignment, particularly reinforcement learning from human feedback (RLHF) in the presence of heterogeneous preferences. We argue that learning a single global reward function is fundamentally insufficient to capture population-level diversity and may violate basic social choice principles. To address this, we present a framework based on committees of reward functions, designed so that pairwise preferences induced by the committee proportionally reflect those of human annotators. We discuss theoretical guarantees showing that small committees suffice to achieve low proportionality error, as well as empirical evidence demonstrating substantial improvements over any single deterministic reward model.

Keywords

Main Subjects

References

Amiri, M. K., Zaferani, S. P. G., Emami, M. R. S., Zahmatkesh, S., Pourhanasa, R., Namaghi, S. S., Klemeš, J., Bokhari, A., & Hajiaghaei-Keshteli, M. (2023). Multi-objective optimization of thermophysical properties of powders-DW/EG NF by RSM, NSGA-II, ANN, MLP and ML. Energy, 280, 128176. https://doi.org/10.1016/j.energy.2023.128176
Anyanwu, K., Mansouri, S., & Adei, D. (2025). Towards declarative blockchains: A SHACL-based model for robust and efficient transactions. In The 2025 IEEE International Conference on Blockchain and Cryptocurrency (ICBC). https://doi.org/10.1109/ICBC64466.2025.11114583
Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Singh Lubana, E., Jenner, E., Casper, S., Sourbut, O., Edelman, B., Zhang, Z., Günther, M., Korinek, A., Hernandez-Orallo, J., Hammond, L., Bigelow, E., Pan, A., Langosco, L., Korbak, T., & … Krueger, D. (2024). Foundational challenges in assuring alignment and safety of large language models. arXiv preprint arXiv:2404.09932. https://doi.org/10.48550/arXiv.2404.09932
Arrow, K. J. (1964). Social choice and individual values (Vol. 2). Wiley.
Aziz, H., Brill, M., Conitzer, V., Elkind, E., Freeman, R., & Walsh, T. (2017). Justified representation in approval-based committee voting. Social Choice and Welfare, 48(2), 461–485.
Aziz, H., Lee, B. E., Morota Chu, S., & Vollen, J. (2024). Proportionally representative clustering. In Proceedings of the 20th Conference on Web and Internet Economics (WINE) (pp. 1075–1083).
Balcan, M.-F., Dick, T., Northgate, R., & Procaccia, A. D. (2019). Envy-free classification. In Proceedings of the 33rd Annual Conference on Neural Information Processing Systems (NeurIPS) (pp. 1238–1248).
Bıyık, E., Huynh, N., Kochenderfer, M. J., & Sadigh, D. (2024). Active preference-based Gaussian process regression for reward learning and optimization. International Journal of Robotics Research, 43(5), 665–684.
Biyik, E., & Sadigh, D. (2018). Batch active preference-based learning of reward functions. In Conference on Robot Learning (pp. 519–528). PMLR.
Caragiannis, I., Micha, E., & Peters, J. (2024). Can a few decide for many? The metric distortion of sortition. In Proceedings of the 41st International Conference on Machine Learning (ICML).
Caragiannis, I., Micha, E., & Shah, N. (2024). Proportional fairness in non-centroid clustering. In Proceedings of the 41st International Conference on Machine Learning (ICML).
Chakraborty, S., Qiu, J., et al. (2024). MaxMin-RLHF: Alignment with diverse human preferences. In Proceedings of the 41st International Conference on Machine Learning (Vol. 235, pp. 6116–6135).
Chen, X., Fain, B., Lyu, L., & Munagala, K. (2019). Proportionally fair clustering. In Proceedings of the 36th International Conference on Machine Learning (ICML) (pp. 1032–1041).
Chen, A., Malladi, S., Zhang, L. H., Chen, X., Zhang, Q., Ranganath, R., & Cho, K. (2024). Preference learning algorithms do not learn preference rankings. In Proceedings of the 37th Annual Conference on Neural Information Processing Systems (NeurIPS) (pp. 101928–101968).
Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., & Amodei, D. (2017). Deep reinforcement learning from human preferences. In Proceedings of the 30th Annual Conference on Neural Information Processing Systems (NeurIPS).
Conitzer, V., Freedman, R., Heitzig, J., Holliday, W. H., Jacobs, B. M., Lambert, N., Mossé, M., et al. (2024). Social choice should guide AI alignment in dealing with diverse human feedback. In Proceedings of the 41st International Conference on Machine Learning (ICML).
Dai, J., & Fleisig, E. (2024). Mapping social choice theory to RLHF [arXiv preprint]. arXiv:2404.13038.
Dragan, A. (2024). Google DeepMind: AI safety …ok doomer: With Anca Dragan [Video]. YouTube. https://www.youtube.com/watch?v=ZXA2dmFxXmg
Dummett, M. (1984). Voting procedures. Oxford University Press.
Dumoulin, V., Johnson, D. D., Castro, P. S., Larochelle, H., & Dauphin, Y. (2024). A density estimation perspective on learning from pairwise human preferences. arXiv preprint arXiv:2311.14115. https://doi.org/10.48550/arXiv.2311.14115
Ebadian, S., & Micha, E. (2025). Boosting sortition via proportional representation. In Proceedings of the 24th International Conference on Autonomous Agents and Multiagent Systems (AAMAS). Forthcoming.
Fairstein, R., Benadè, G., & Gal, K. (2023). Participatory budgeting designs for the real world. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI) (pp. 5633–5640).
Faliszewski, P., Skowron, P., Slinko, A., & Talmon, N. (2017). Multiwinner voting: A new challenge for social choice theory. Trends in Computational Social Choice, 74(2017), 27–47.
Farahani, R. Z., & Hekmatfar, M. (2009). Facility location: Concepts, models, algorithms and case studies. Springer Science & Business Media.
Farahani, R. Z., SteadieSeifi, M., & Asgari, N. (2010). Multiple criteria facility location problems: A survey. Applied Mathematical Modelling, 34(7), 1689–1709.
Feng, S., Sorensen, T., Liu, Y., Fisher, J., Park, C. Y., Choi, Y., & Tsvetkov, Y. (2024). Modular pluralism: Pluralistic alignment via multi-LLM collaboration. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 4151–4171).
Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning (pp. 1126–1135). PMLR.
Freeman, R., Micha, E., & Shah, N. (2021). Two-sided matching meets fair division. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21) (pp. 203–209).
Gambhir, M. (2026). A brief literature review and summary of reinforcement learning with human feedback.
Ge, L., et al. (2024). Axioms for AI alignment from human feedback. Advances in Neural Information Processing Systems, 37, 80439–80465.
Halpern, D., Micha, E., Procaccia, A., & Shapira, I. (2026). Pairwise calibrated rewards for pluralistic alignment. Advances in Neural Information Processing Systems, 38, 57882–57916.
Hejna, D. J., & Sadigh, D. (2023). Few-shot preference learning for human-in-the-loop RL. In Conference on Robot Learning (pp. 2014–2025). PMLR. https://doi.org/10.48550/arXiv.2212.03363
Hua, Y., Na, H., Li, Z., Liu, F., Fang, X., Clifton, D., & Torous, J. (2025). A scoping review of large language models for generative tasks in mental health care. npj Digital Medicine, 8(1), 230.
Kalayci, Y. H., Kempe, D., & Kher, V. (2024). Proportional representation in metric spaces and low-distortion committee selection. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI) (pp. 9815–9823).
Kellerhals, L., & Peters, J. (2024). Proportional fairness in clustering: A social choice perspective. In Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS).
Khalifa, M., Elsahar, H., & Dymetman, M. (2021). A distributional approach to controlled text generation. In Proceedings of the 9th International Conference on Learning Representations (ICLR).
Kirk, H. R., Vidgen, B., Röttger, P., & Hale, S. A. (2024a). The benefits, risks and bounds of personalizing the alignment of large language models to individuals. Nature Machine Intelligence, 6(4), 383–392.
Kirk, R., Mediratta, I., Nalmpantis, C., Luketina, J., Hambro, E., Grefenstette, E., & Raileanu, R. (2024b). Understanding the effects of RLHF on LLM generalisation and diversity. In Proceedings of the 12th International Conference on Learning Representations (ICLR).
Kupcsik, A., Hsu, D., & Lee, W. S. (2018). Learning dynamic robot-to-human object handover from human feedback. In Robotics Research: Volume 1 (pp. 161–176). Springer.
Lackner, M., & Skowron, P. (2022). Approval-based committee voting. In Multi-winner voting with approval preferences (pp. 1–7). Springer.
Lai, F., Zhu, X., Madhyastha, H. V., & Chowdhury, M. (2021). Oort: Efficient federated learning via guided participant selection. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21) (pp. 19–35).
Lake, T., Choi, E., & Durrett, G. (2025). From distributional to overton pluralism: Investigating large language model alignment. In Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) (pp. 6794–6814).
Li, B., Li, L., Sun, A., Wang, C., & Wang, Y. (2021). Approximate group fairness for clustering. In Proceedings of the 38th International Conference on Machine Learning (ICML) (pp. 6381–6391).
Mahpouya, F., Burris, C. J., Paul, H., & Nikolaev, A. (2026). Maximizing the expected value of experimentation for finding top-κ rank via aggregation of pairwise comparisons. IISE Transactions, 1–20.
Mansouri, S., Mohammed, H., Korchiev, N., & Anyanwu, K. (2024). Taming smart contracts with blockchain transaction primitives: A possibility? In the 2024 IEEE International Conference on Blockchain (Blockchain).
Micha, E., & Shah, N. (2020). Proportionally fair clustering revisited. In Proceedings of the 47th International Colloquium on Automata, Languages, and Programming (ICALP) (Article 85, pp. 1–16).
Mukherjee, A., et al. (2026). SharedRep-RLHF: A shared representation approach to RLHF with diverse preferences. In Proceedings of the AAAI Conference on Artificial Intelligence.
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022). Training language models to follow instructions with human feedback. In Proceedings of the 35th Annual Conference on Neural Information Processing Systems (NeurIPS) (pp. 27730–27744).
Park, C., Liu, M., Kong, D., Zhang, K., & Ozdaglar, A. (2024). RLHF from heterogeneous feedback via personalization and preference aggregation. arXiv:2405.00254.
Peters, D. (2024). Proportional representation for artificial intelligence. In 27th European Conference on Artificial Intelligence. IOS Press.
Peters, D., Pierczyński, G., & Skowron, P. (2021). Proportional participatory budgeting with additive utilities. In Proceedings of the 34th Annual Conference on Neural Information Processing Systems (NeurIPS) (pp. 12726–12737).
Pezeshgi, A., Abarghoei, M. V., Naeimi, M., & Family, Q. (2026). Trusting the machine: How consumer trust in artificial intelligence shapes future adoption intentions. Management Science Advances, 3(1), 237–245.
Pezeshgi, A., Naeimi, M., & Pezeshgi, Q. (2025). Buying on impulse in the age of AI: Mechanisms, evidence, and moral dilemmas. Evidence, and Moral Dilemmas (August 07, 2025).
Peters, D., & Skowron, P. (2020). Proportionality and the limits of welfarism. In Proceedings of the 21st ACM Conference on Economics and Computation (pp. 793–794).
Poddar, S., Wan, Y., Ivison, H., Gupta, A., & Jaques, N. (2024). Personalizing reinforcement learning from human feedback with variational preference learning. In Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS).
Procaccia, A., Schiffer, B., Wang, S., & Zhang, S. (2025). Metritocracy: Representative metrics for lite benchmarks [arXiv preprint]. arXiv:2506.09813.
Revel, M., Milli, S., Lu, T., Watson-Daniels, J., & Nickel, M. (2025). Representative ranking for deliberation in the public sphere. In Proceedings of the 42nd International Conference on Machine Learning (ICML).
Safizadeh, M., Yazdanparast, A., & Felix, R. (2026). Taking pride in vegan consumption: A construal level theory account of ad message appeal and future self connectedness. Psychology & Marketing.
Sahoo, S., et al. (2025). Position: The complexity of perfect AI alignment—Formalizing the RLHF trilemma [arXiv preprint]. arXiv:2511.19504.
Sattler, F., Müller, K.-R., & Samek, W. (2020). Clustered federated learning: Model-agnostic distributed multitask optimization under privacy constraints. IEEE Transactions on Neural Networks and Learning Systems, 32(8), 3710–3722.
Shi, Y., Liu, Z., Shi, Z., & Yu, H. (2023). Fairness-aware client selection for federated learning. In 2023 IEEE International Conference on Multimedia and Expo (ICME) (pp. 324–329). IEEE.
Shypula, A., Li, S., Zhang, B., Padmakumar, V., Yin, K., & Bastani, O. (2025). Evaluating the diversity and quality of LLM generated content [arXiv preprint]. arXiv:2504.12522.
Small, C., Bjorkegren, M., Erkkilä, T., Shaw, L., & Megill, C. (2021). Polis: Scaling deliberation by mapping high dimensional opinion spaces. Recerca: Revista de Pensament i Anàlisi, 26(2), 1–26.
Sorensen, T., Moore, J., Fisher, J., Gordon, M., Mireshghallah, N., Rytting, C. M., Ye, A., Jiang, L., Lu, X., Dziri, N., Althoff, T., & Choi, Y. (2024). A roadmap to pluralistic alignment. arXiv preprint arXiv:2402.05070. https://doi.org/10.48550/arXiv.2402.05070
Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., & Christiano, P. (2020). Learning to summarize from human feedback. In Proceedings of the 31st Annual Conference on Neural Information Processing Systems (NeurIPS) (pp. 3008–3021).
Swamy, G., Dann, C., Kidambi, R., Wu, Z. S., & Agarwal, A. (2024). A minimaximalist approach to reinforcement learning from human feedback. In Proceedings of the 41st International Conference on Machine Learning (ICML).
Viappiani, P., & Boutilier, C. (2010). Optimal Bayesian recommendation sets and myopically optimal choice query sets. In Proceedings of the 6th Annual Conference on Neural Information Processing Systems (NeurIPS) (pp. 2352–60).
Wang, H., Z. Kaplan, D. Niu, & Li, B. (2020). Optimizing federated learning on non-iid data with reinforcement learning. In IEEE INFOCOM 2020-IEEE Conference on Computer Communications (pp. 1698–707). IEEE.
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P. S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., Kenton, Z., Brown, S., Hawkins, W., Stepleton, T., Biles, C., Birhane, A., Haas, J., Rimell, L., Hendricks, L.,... & Gabriel, I. (2021). Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359.
Yu, T., Quillen, D., He, Z., Julian, R., Hausman, K., Finn, C., & Levine, S. (2020, May). Meta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning (pp. 1094-1100). PMLR.
Zhang, J., Lin, Y., Lin, M., & Liu, J. (2016). An effective collaborative filtering algorithm based on user preference clustering. Applied Intelligence, 45(2), 230-240.
Zhong, H., Deng, Z., Su, W. J., Wu, Z. S., & Zhang, L. (2024). Provable multi-party reinforcement learning with diverse human feedback. arXiv preprint arXiv:2403.05006.
Zhu, B., Jordan, M., & Jiao, J. (2023, July). Principled reinforcement learning with human feedback from pairwise or k-wise comparisons. In International Conference on Machine Learning (pp. 43037-43067). PMLR.
Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., & Irving, G. (2019). Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593. https://doi.org/10.48550/arXiv.1909.08593
Jafari, M., Akhavan, P., & Akbari, A. H. (2026). Enhancing supply chain agility and performance through big data analytics: the role of digitalization and top management support. International Journal of Productivity and Performance Management, 1-22. https://doi.org/10.1108/IJPPM-06-2025-0557 
Tavakkoli-Moghaddam, R., Akbari, A. H., Tanhaeean, M., Moghdani, R., Gholian-Jouybari, F., & Hajiaghaei-Keshteli, M. (2024). Multi-objective boxing match algorithm for multi-objective optimization problems. Expert Systems with Applications, 239, 122394. https://doi.org/10.1016/j.eswa.2023.122394 

Files

Share

How to cite

Statistics