Large Language Model vs. Stack Overflow in Addressing Android Permission Related Challenges
The Android permission system regulates access to sensitive mobile device resources and data, such as camera and location. To access these resources, third-party application (app) developers need to request permissions. However, the Android permission system is complex and fast-evolving, presenting developers with numerous challenges surrounding compatibility issues, misuse of permissions, and vulnerabilities related to permissions. Our study aims to explore whether Large Language Models (LLMs) can serve as a reliable tool to assist developers in using Android permissions correctly and securely, thereby reducing the risks of misuse and security vulnerabilities in apps. In our study, we analyzed 1,008 Stack Overflow questions related to Android permissions and their accepted answers. In parallel, we generate answers to these questions using a popular LLM tool, ChatGPT. In our evaluation, we focus on how well the AI-generated responses align with the accepted answers provided by developers on Stack Overflow. Our findings show that above 50% of ChatGPT-provided answers align with Stack Overflow’s accepted answers. ChatGPT offers better-aligned responses for challenges related to Documentation and Conceptual Understanding, while it provides less aligned answers for Debugging-related issues. In addition, we found that ChatGPT provides more consistent answers for Documentation and Conceptual Understanding, while Implementation-related answers are the least consistent. Our study demonstrates the potential for using LLMs such as ChatGPT as a supporting tool to help developers navigate Android permission-related problems.
Mon 15 AprDisplayed time zone: Lisbon change
16:00 - 17:30 | Mobile AppsData and Tool Showcase Track / Technical Papers at Almada Negreiros Chair(s): Dario Di Nucci University of Salerno | ||
16:00 12mTalk | Automating GUI-based Test Oracles for Mobile Apps Technical Papers Kesina Baral CQSE America, Jack Johnson , Junayed Mahmud George Mason University, Sabiha Salma George Mason University, Mattia Fazzini University of Minnesota, Julia Rubin University of British Columbia, Jeff Offutt George Mason University, Kevin Moran University of Central Florida | ||
16:12 12mTalk | Global Prosperity or Local Monopoly? Understanding the Geography of App Popularity Technical Papers Liu Wang Beijing University of Posts and Telecommunications, Conghui Zheng Beijing University of Posts and Telecommunications, Haoyu Wang Huazhong University of Science and Technology, Xiapu Luo The Hong Kong Polytechnic University, Gareth Tyson Queen Mary University of London, Yi Wang , Shangguang Wang Beijing University of Posts and Telecommunications | ||
16:24 12mTalk | GuiEvo: Automated Evolution of Mobile App UIs Technical Papers Sabiha Salma George Mason University, S M Hasan Mansur George Mason University, Yule Zhang George Mason University, Kevin Moran University of Central Florida | ||
16:36 12mTalk | Comparing Apples to Androids: Discovery, Retrieval, and Matching of iOS and Android Apps for Cross-Platform Analyses Technical Papers Magdalena Steinböck TU Wien, Jakob Bleier TU Wien, Mikka Rainer CISPA Helmholtz Center for Information Security, Tobias Urban Institute for Internet Security & secunet Security Networks AG, Christine Utz CISPA Helmholtz Center for Information Security, Martina Lindorfer TU Wien | ||
16:48 12mTalk | Keep Me Updated: An Empirical Study on Embedded Javascript Engines in Android Apps Technical Papers Elliott Wen The University of Auckland, Jiaxiang Liu The Hong Kong Polytechnic University, Xiapu Luo The Hong Kong Polytechnic University, Giovanni Russello University of Auckland, Jens Dietrich Victoria University of Wellington | ||
17:00 12mTalk | Large Language Model vs. Stack Overflow in Addressing Android Permission Related Challenges Technical Papers Sahrima Jannat Oishwee University of Saskatchewan, Natalia Stakhanova University of Saskatchewan, Zadia Codabux University of Saskatchewan, Canada | ||
17:12 4mTalk | DATAR: A Dataset for Tracking App Releases Data and Tool Showcase Track Yasaman Abedini Sharif University of Technology, Mohammad Hadi Hajihosseini Sharif University of Technology, Abbas Heydarnoori Bowling Green State University | ||
17:16 4mTalk | AndroZoo: A Retrospective with a Glimpse into the Future Data and Tool Showcase Track Marco Alecci University of Luxembourg, Pedro Jesús Ruiz Jiménez University of Luxembourg, Kevin Allix Independent Researcher, Tegawendé F. Bissyandé University of Luxembourg, Jacques Klein University of Luxembourg |