Evaluating Cultural Hallucinations in Multimodal Generative AI: A Transparency and Trustworthiness Perspective
Keywords:
cultural hallucination, multimodal generative AI, transparency, trustworthiness, cultural representation, text-to-image generation, algorithmic auditing, socio-technical systemsAbstract
The rapid advancement of multimodal generative artificial intelligence has introduced capabilities for synthesizing images, text, and video with unprecedented fidelity. While substantial effort has focused on factual hallucinations in language models and object-level hallucinations in vision-language systems, a more insidious class of failure remains underexamined: cultural hallucination, defined as the systematic distortion, stereotyping, or erasure of cultural symbols, practices, and visual identities. This paper provides a system-level analysis of cultural hallucinations through the intertwined lenses of transparency and trustworthiness. We argue that cultural hallucinations arise not merely from insufficient data representation but from a confluence of architectural design choices, opaque curation pipelines, and evaluation protocols that prioritize object-centric accuracy over culturally grounded semantic fidelity. Drawing on interdisciplinary literature spanning machine learning, science and technology studies, and algorithmic auditing, we examine how contrastive pre-training objectives, dataset homogenization, and prompt-to-image decoding amplify hegemonic visual narratives. We further assess the limitations of existing transparency instruments, including model cards and datasheets, in capturing culturally specific failure modes, and propose extensions that incorporate culture-aware auditing, participatory red-teaming, and interpretability probes sensitive to symbolic meaning. Deployment-level consequences are analyzed, highlighting how cultural hallucinations erode public trust, perpetuate symbolic marginalization, and constrain the safe integration of generative models into education, media, and cross-cultural communication. We conclude by outlining a governance and research infrastructure framework that treats cultural veracity as a first-class dimension of model trustworthiness, calling for multi-stakeholder observatories, culturally stratified evaluation benchmarks, and continuous monitoring mechanisms that transcend static accuracy metrics.
References
1. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21) (pp. 610–623). ACM. https://doi.org/10.1145/3442188.3445922
2. Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., ... & Gabriel, I. (2021). Ethical and social risks of harm from language models. arXiv preprint arXiv:2112.04359.
3. Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., ... & Gebru, T. (2019). Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT ’19) (pp. 220–229). ACM. https://doi.org/10.1145/3287560.3287596
4. Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., ... & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT ’20) (pp. 33–44). ACM. https://doi.org/10.1145/3351095.3372873
5. Birhane, A., Prabhu, V. U., & Kahembwe, E. (2021). Multimodal datasets: Misogyny, pornography, and malignant stereotypes. arXiv preprint arXiv:2110.01963.
6. Luccioni, A. S., & Viviano, J. D. (2021). What’s in the box? An analysis of the LAION-400M dataset. arXiv preprint arXiv:2111.02114.
7. Cho, J., Zala, A., & Bansal, M. (2022). DALL-Eval: Probing the reasoning skills and social biases of text-to-image generative transformers. arXiv preprint arXiv:2202.04053.
8. Wolfe, R., & Caliskan, A. (2022). Markedness in visual semantic AI. In 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’22) (pp. 1269–1279). ACM. https://doi.org/10.1145/3531146.3533174
9. Bianchi, F., Kalluri, P., Durmus, E., Ladhak, F., Cheng, M., Nozza, D., ... & Hashimoto, T. (2023). Easily accessible text-to-image generation amplifies demographic stereotypes at large scale. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’23) (pp. 1493–1504). ACM. https://doi.org/10.1145/3593013.3594095
10. Struppek, L., Hintersdorf, D., Friedrich, F., Brack, M., Schramowski, P., & Kersting, K. (2022). Exploiting cultural biases via homoglyphs in text-to-image synthesis. arXiv preprint arXiv:2209.08891.
11. Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, W. X., & Wen, J.-R. (2023). Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 2925–2941). Association for Computational Linguistics.
12. Dai, W., Liu, Z., Ji, Z., Su, D., & Fung, P. (2023). HallusionBench: You see what you think? Or you think what you see? An advanced diagnostic benchmark for multimodal hallucination. arXiv preprint arXiv:2310.14566.
13. Rohrbach, A., Hendricks, L. A., Burns, K., Darrell, T., & Saenko, K. (2018). Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 4035–4045). Association for Computational Linguistics. https://doi.org/10.18653/v1/D18-1437
14. Biten, A. F., Gomez, L., & Karatzas, D. (2022). Let there be a clock on the beach: Reducing object hallucination in vision-language models. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (pp. 1381–1390). IEEE.
15. Prabhu, V. U., & Birhane, A. (2020). Large image datasets: A pyrrhic win for computer vision? arXiv preprint arXiv:2006.16923.
16. Hutchinson, B., Smart, A., Hanna, A., Denton, E., Greer, C., Kjartansson, O., ... & Mitchell, M. (2021). Towards accountability for machine learning datasets: Practices from software engineering and infrastructure. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’21) (pp. 560–575). ACM. https://doi.org/10.1145/3442188.3445918
17. Raji, I. D., & Buolamwini, J. (2019). Actionable auditing: Investigating the impact of publicly naming biased performance results of commercial AI products. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’19) (pp. 429–435). ACM. https://doi.org/10.1145/3306618.3314244
18. C. Shi, S. Li, S. Guo, S. Xie, W. Wu, J. Dou, C. Wu, C. Xiao, C. Wang, Z. Cheng, et al. (2025)Where culture fades: revealing the cultural gap in text-to-image generation.arXiv preprint arXiv:2511.17282.
19. Floridi, L., Cowls, J., Beltrametti, M., Chatila, R., Chazerand, P., Dignum, V., ... & Schafer, B. (2018). AI4People—An ethical framework for a good AI society: Opportunities, risks, principles, and recommendations. Minds and Machines, 28(4), 689–707. https://doi.org/10.1007/s11023-018-9482-5
20. Noble, S. U. (2018). Algorithms of oppression: How search engines reinforce racism. New York University Press.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 International Journal of Artificial Intelligence Engineering and Systems
This work is licensed under a Creative Commons Attribution 4.0 International License.
This article is published under the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
