Integrating AI into Multimodal Automotive Design: A Conceptual Framework for User Experience Evaluation and Market Application

Mu Liyuan; Sharfika Raine; Shi Zhehan

doi:10.56982/dream.v5i01.324

Authors

Mu Liyuan City University, Kuala Lumpur, Malaysia
Sharfika Raine City University, Kuala Lumpur, Malaysia
Shi Zhehan City University, Kuala Lumpur, Malaysia

DOI:

https://doi.org/10.56982/dream.v5i01.324

Keywords:

AI Technology, User Experience, Evaluation and Market Application, Multi -Modal Interaction Design, Automotive

Abstract

The rapid evolution of artificial intelligence (AI) and multimodal interaction technologies is reshaping automotive design, demanding new frameworks that prioritize user experience (UX) and market applicability. This conceptual study proposes an integrative framework that combines AI-driven personalization, multimodal interface design (e.g., voice, gesture, and touch), and real-time UX evaluation mechanisms. Drawing upon human-centered design principles and theories of user acceptance, the framework addresses current gaps in adaptive, intelligent vehicle interface systems. It further outlines strategic pathways for deployment in diverse market environments through an evaluation model that accounts for technological scalability, cultural preferences, and demographic diversity. The study concludes by identifying key directions for future research, particularly emphasizing cross-cultural UX testing across various vehicle types and user groups. The proposed framework contributes to both academic discourse and industry practice, offering a foundation for the next generation of intelligent, user-centric automotive systems.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

References

Alonso, J., Bernal, J., & Ruiz, M. (2022). AI-driven personalization in automotive interfaces. IEEE Transactions on Intelligent Transportation Systems, 23(7), 12452–12465. https://doi.org/10.1109/TITS.2022.3145690

Arya, V., Pandey, N., Paul, J., & Sinha, P. K. (2022). Exploring AI transparency for improved customer trust and experience. Journal of Business Research, 139, 1392–1402. https://doi.org/10.1016/j.jbusres.2021.10.051

Bello, I., Olivares, R., & Chang, R. (2021). Evaluating user experience in connected vehicles: A UX-centric approach. International Journal of Human–Computer Studies, 148, 102590. https://doi.org/10.1016/j.ijhcs.2021.102590

BMW Group. (2023). How AI is revolutionising production. Retrieved from https://www.bmwgroup.com/en/news/general/2023/aiqx.html

Brooke, J. (1996). SUS: A quick and dirty usability scale. In P. W. Jordan et al. (Eds.), Usability Evaluation in Industry (pp. 189–194). Taylor & Francis.

Chen, R., Liu, J., & Zhang, T. (2023). AI-enhanced automotive design: Challenges and opportunities in user-centered innovation. International Journal of Human–Computer Interaction, 39(1), 58–76. https://doi.org/10.1080/10447318.2022.2109873

Chen, Y., Wang, H., & Zhou, L. (2022). Smart vehicle systems: The role of computer vision in multimodal interaction. IEEE Transactions on Intelligent Vehicles, 7(1), 45–58. https://doi.org/10.1109/TIV.2022.3142245

Choi, H., Jeong, H., & Kim, Y. (2023). Empirical assessment of user acceptance in autonomous vehicle human–machine interfaces. International Journal of Human–Computer Studies, 175, 103092. https://doi.org/10.1016/j.ijhcs.2023.103092

Ehsan, U., & Riedl, M. O. (2021). Explainability in human–AI interaction: Designing for effective communication. ACM Transactions on Interactive Intelligent Systems, 11(2), 1–42. https://doi.org/10.1145/3397481 DOI: https://doi.org/10.1145/3411764.3445188

Fang, R., Chen, Y., & Liu, S. (2023). Enhancing human-machine interaction in vehicles using AI-powered multimodal systems. IEEE Transactions on Human-Machine Systems, 53(1), 25–36. https://doi.org/10.1109/THMS.2023.3264412

Georgiev, G. V., Taura, T., & Nagai, Y. (2020). The transition from conventional to intelligent design practices in automotive engineering. Design Science, 6, e2. https://doi.org/10.1017/dsj.2020.2

Han, J., & Lee, S. (2022). Applying conjoint analysis to evaluate customer preferences for AI-enabled automotive systems. International Journal of Market Research, 64(1), 89–107. https://doi.org/10.1177/14707853211039213

He, J., Zhang, Y., & Wu, T. (2022). Intelligent interface design for autonomous vehicles: A user-centered AI approach. International Journal of Human–Computer Studies, 164, 102828. https://doi.org/10.1016/j.ijhcs.2022.102828

ISO. (2019). Ergonomics of human-system interaction – Part 210: Human-centred design for interactive systems (ISO 9241-210:2019). International Organization for Standardization.

International Council on Clean Transportation. (2023). Electric vehicle market and policy developments in U.S. states, 2023. Retrieved from https://theicct.org/publication/ev-ldv-us-major-markets-monitor-2023-june24/

International Energy Agency. (2024). Global EV Outlook 2024 – Trends in electric cars. Retrieved from https://www.iea.org/reports/global-ev-outlook-2024/trends-in-electric-cars

Jäger, M., Pion, C., & Kiesel, A. (2022). Cognitive load and learning curves in multimodal in-vehicle interactions. Human Factors, 64(5), 849–862. https://doi.org/10.1177/00187208211024732

Jeong, D., Kang, Y., & Lee, K. (2022). A real-time driver emotion recognition system using facial expression and physiological data fusion. Sensors, 22(4), 1503. https://doi.org/10.3390/s22041503

Kim, S., Park, J., & Lee, H. (2022). Human-centered AI applications in automotive UX design: A review and future directions. Applied Sciences, 12(14), 6983. https://doi.org/10.3390/app12146983

Kim, J., Lee, S., & Park, Y. (2023). Assessing user-centered design in autonomous vehicle interfaces: Impacts on perceived usefulness and satisfaction. International Journal of Human–Computer Interaction, 39(2), 151–165. https://doi.org/10.1080/10447318.2022.2086843

Kim, J., Park, H., & Choi, Y. (2023). Consumer acceptance of AI-based voice assistants in smart cars: A choice-based conjoint approach. Journal of Retailing and Consumer Services, 74, 103300. https://doi.org/10.1016/j.jretconser.2023.103300

Körber, M. (2023). Validating human-machine interaction in automated vehicles: Insights from in-lab and on-road studies. Human Factors, 65(2), 215–228. https://doi.org/10.1177/00187208221088867

Lee, S. J., Kim, H., & Park, M. (2021). Extending the Technology Acceptance Model (TAM) to AI-based multimodal interfaces in smart vehicles. Computers in Human Behavior, 117, 106648. https://doi.org/10.1016/j.chb.2020.106648

Lee, H., & Kim, D. (2023). Smart mobility and user-centric design: Integrating AI and multimodal feedback. Design Studies, 83, 101157. https://doi.org/10.1016/j.destud.2023.101157

Li, S., & Yang, J. (2021). Haptic feedback for vehicular interfaces: A review of technologies and applications. International Journal of Human–Computer Studies, 153, 102654. https://doi.org/10.1016/j.ijhcs.2021.102654

Liao, Y., Tang, J., & Zhang, H. (2023). Exploring generative AI in automotive product design: Enhancing creativity and efficiency. Design Studies, 84, 102145. https://doi.org/10.1016/j.destud.2022.102145

Lin, J., & Wang, H. (2023). Designing for driving: An activity theory and affordance-based approach to gesture interaction in smart cars. International Journal of Human–Computer Studies, 175, 102993. https://doi.org/10.1016/j.ijhcs.2023.102993

Liu, R., Zhang, L., & Tan, Z. (2023). Adaptive multimodal interfaces for in-vehicle HMI: A deep learning approach. IEEE Transactions on Vehicular Technology, 72(2), 1561–1574. https://doi.org/10.1109/TVT.2023.3237654

Liu, H., Zhang, Q., & Chen, Y. (2022). Evaluating multimodal automotive HMI performance in lab and real-world settings. International Journal of Human-Computer Studies, 163, 102823. https://doi.org/10.1016/j.ijhcs.2022.102823

Li, X., Xu, Y., & Wang, J. (2023). Cultural adaptation of in-vehicle AI systems: Toward localized UX design. Human-Centric Computing and Information Sciences, 13, 14. https://doi.org/10.1186/s13673-023-00303-9

Lumenalta. (2024). 11 Gen AI Use Cases in the Automotive Industry. Retrieved from https://lumenalta.com/insights/11-gen-ai-use-cases-in-automotive-industry

Meyer, T., Hoffmann, A., & Zhao, R. (2022). User experience as a driver of customer loyalty in smart mobility solutions. Journal of Product Innovation Management, 39(5), 789–804. https://doi.org/10.1111/jpim.12555

Meschtscherjakov, A., Wilfinger, D., & Tscheligi, M. (2020). Towards a unified user experience model for automotive HMIs. Proceedings of the 12th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 174–183. https://doi.org/10.1145/3409120.3410640

Mun, H., Kwon, J., & Kim, S. (2023). AI-enabled generative design in automotive interiors: A case study on UX-driven modeling. International Journal of Industrial Ergonomics, 96, 103456. https://doi.org/10.1016/j.ergon.2023.103456

NVIDIA. (2023). BYD Automotive Partner. Retrieved from https://www.nvidia.com/en-us/self-driving-cars/partners/byd/

Norman, D. A. (2013). The design of everyday things (Rev. ed.). MIT Press.

Palinko, O., Kun, A. L., Shyrokov, A., & Heeman, P. A. (2020). Exploring visual attention of drivers with eye tracking in a naturalistic setting. Transportation Research Part F: Traffic Psychology and Behaviour, 68, 36–48. https://doi.org/10.1016/j.trf.2019.11.012

Park, J., Choi, Y., & Kim, S. (2022). Context-aware multimodal interfaces in autonomous vehicles. International Journal of Human–Computer Interaction, 38(11), 1043–1056. https://doi.org/10.1080/10447318.2021.1981452

Peffers, K., Rothenberger, M., & Kuechler, B. (2007). A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 45–77. https://doi.org/10.2753/MIS0742-1222240302 DOI: https://doi.org/10.2753/MIS0742-1222240302

PwC. (2023). Smart Mobility Hub. Retrieved from https://www.pwc.com/gx/en/issues/reinventing-the-future/smart-mobility-hub.html

PwC Strategy&. (2023). Digital Auto Report 2023 (Volume 2). Retrieved from https://www.strategyand.pwc.com/de/en/industries/automotive/digital-auto-report/volume2.html

Rahman, S. A., & Ng, C. K. (2022). Voice-AI in electric vehicles: Trust and explainability as antecedents of technology acceptance. Journal of Intelligent & Robotic Systems, 104(2), 18. https://doi.org/10.1007/s10846-021-01400-1

Ribeiro, M. T., Singh, S., & Guestrin, C. (2020). “Why should I trust you?” Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD, 1135–1144. https://doi.org/10.1145/2939672.2939778 DOI: https://doi.org/10.1145/2939672.2939778

Rümelin, S., Butz, A., & Heuten, W. (2020). Gaze-based interaction in the car: A review of safety, usability, and user perception. AutomotiveUI Conference Proceedings, 112–120. https://doi.org/10.1145/3409118.3409155

Schrepp, M., Hinderks, A., & Thomaschewski, J. (2021). Design and evaluation of a short version of the user experience questionnaire (UEQ-S). International Journal of Interactive Multimedia and Artificial Intelligence, 6(2), 103–108. https://doi.org/10.9781/ijimai.2021.02.005 DOI: https://doi.org/10.9781/ijimai.2017.09.001

Schneider, T., Müller, M., & Klein, R. (2023). Trusting the AI in your car: The role of transparency in automotive recommendation systems. Transportation Research Part F: Traffic Psychology and Behaviour, 91, 185–199. https://doi.org/10.1016/j.trf.2023.01.014

Shaout, A., & Jarrah, M. (2014). Integrating infotainment and telematics in automotive systems: A review. International Journal of Engineering and Technology, 6(3), 236–245.

Shao, H., Li, Y., & Xu, F. (2022). Designing multimodal in-vehicle interfaces: A user-centric review. Applied Ergonomics, 101, 103696. https://doi.org/10.1016/j.apergo.2021.103696 DOI: https://doi.org/10.1016/j.apergo.2022.103696

Shin, D., Choi, M., & Lee, H. (2022). UX design strategies for next-generation vehicles: A user-centric approach in smart cockpit development. Design Studies, 79, 101083. https://doi.org/10.1016/j.destud.2021.101083

Sjöbergh, J., & Alvemo, S. (2022). Emotion-aware systems in automotive design: Improving driver interaction through AI. ACM Transactions on Human-Robot Interaction, 11(4), 22. https://doi.org/10.1145/3554562

Sun, M., Zhang, Y., & Lin, Q. (2023). Adaptive multimodal interaction for intelligent vehicles: A user-centric design perspective. ACM Transactions on Interactive Intelligent Systems, 13(1), 1–25. https://doi.org/10.1145/3581234 DOI: https://doi.org/10.1145/3558772

Tostado, J. J., Ramos, J., & Suárez, A. (2023). Adaptive user experience frameworks for AI-based automotive systems. Human–Computer Interaction, 38(3), 210–229. https://doi.org/10.1080/07370024.2022.2158792

Vella, D., & Enke, J. (2021). Designing future mobility: UX and intelligent system integration in autonomous vehicles. Journal of Automotive Design & Technology, 9(2), 112–125.

Venkatesh, V., Thong, J. Y. L., & Xu, X. (2022). A unified framework of technology adoption and use: Revisited and updated. MIS Quarterly, 46(1), 1–42. https://doi.org/10.25300/MISQ/2022/13829

Virtue Market Research. (2024). Smart Mobility Market Size (2024–2030). Retrieved from https://virtuemarketresearch.com/report/smart-mobility-market

Wang, Y., Sun, M., & Zhao, Y. (2021). Evaluating multimodal systems for enhanced driver-vehicle interaction. Transportation Research Part F: Traffic Psychology and Behaviour, 76, 1–14. https://doi.org/10.1016/j.trf.2020.10.009

Wang, L., Zhang, Y., & He, J. (2023). Understanding the role of user experience in the acceptance of intelligent vehicles: An empirical investigation. Transportation Research Part F: Traffic Psychology and Behaviour, 94, 67–78. https://doi.org/10.1016/j.trf.2023.01.005

Wang, Z., Li, Y., & Wang, M. (2023). Evaluating user experience in autonomous vehicle interfaces: A review of methods and metrics. International Journal of Human-Computer Studies, 173, 103127. https://doi.org/10.1016/j.ijhcs.2023.103127

Wang, Y., Li, F., & Jiang, H. (2021). Biometric-based UX evaluation in automotive multimodal systems: A physiological computing perspective. Computers in Human Behavior, 124, 106942. https://doi.org/10.1016/j.chb.2021.106942

Wang, H., Zhao, Q., & Lin, F. (2022). Real-time adaptive interfaces in intelligent cars: AI and multimodality in action. ACM Transactions on Interactive Intelligent Systems, 12(4), 42:1–42:21. https://doi.org/10.1145/3547246

Yin, M., Zhang, T., & Luo, X. (2023). Conversational AI in automotive interfaces: Enhancing driver engagement and safety. ACM Transactions on Interactive Intelligent Systems, 13(2), 1–24. https://doi.org/10.1145/3581312

Zhang, Y., Liu, D., & Chen, X. (2022). Integrating human-centered design into AI-enhanced in-vehicle systems: A UX evaluation framework. IEEE Transactions on Human-Machine Systems, 52(1), 15–26. https://doi.org/10.1109/THMS.2022.3141894

Zhang, X., Wang, Y., & Lu, M. (2023). Reducing driver workload through AI-based multimodal interaction design. Transportation Research Part F: Traffic Psychology and Behaviour, 93, 130–142. https://doi.org/10.1016/j.trf.2023.03.005

Zhao, X., Zhou, Y., & Liu, D. (2023). Cognitive load assessment using eye-tracking and GSR in smart vehicle interfaces. IEEE Transactions on Human-Machine Systems, 53(3), 292–302. https://doi.org/10.1109/THMS.2023.3239380

Zhou, K., Li, X., & Han, Y. (2023). Multimodal interaction in smart vehicles: Challenges and prospects. Human–Computer Interaction, 38(2), 139–157. https://doi.org/10.1080/07370024.2022.2104567

Zion Market Research. (2024). Artificial Intelligence (AI) in Automotive Market. Retrieved from https://www.zionmarketresearch.com/report/artificial-intelligence-automotive-market