1. bookVolume 2017 (2017): Issue 4 (October 2017)
Journal Details
First Published
16 Apr 2015
Publication timeframe
4 times per year
access type Open Access

To Permit or Not to Permit, That is the Usability Question: Crowdsourcing Mobile Apps’ Privacy Permission Settings

Published Online: 10 Oct 2017
Page range: 119 - 137
Received: 28 Feb 2017
Accepted: 02 Jun 2017
Journal Details
First Published
16 Apr 2015
Publication timeframe
4 times per year

Millions of apps available to smartphone owners request various permissions to resources on the devices including sensitive data such as location and contact information. Disabling permissions for sensitive resources could improve privacy but can also impact the usability of apps in ways users may not be able to predict. We study an efficient approach that ascertains the impact of disabling permissions on the usability of apps through large-scale, crowdsourced user testing with the ultimate goal of making recommendations to users about which permissions can be disabled for improved privacy without sacrificing usability.

We replicate and significantly extend previous analysis that showed the promise of a crowdsourcing approach where crowd workers test and report back on various configurations of an app. Through a large, between-subjects user experiment, our work provides insight into the impact of removing permissions within and across different apps (our participants tested three apps: Facebook Messenger (N=218), Instagram (N=227), and Twitter (N=110)). We study the impact of removing various permissions within and across apps, and we discover that it is possible to increase user privacy by disabling app permissions while also maintaining app usability.


[1] Yuvraj Agarwal and Malcolm Hall. 2013. ProtectMyPrivacy: detecting and mitigating privacy leaks on iOS devices using crowdsourcing. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services. ACM, 97–110.Search in Google Scholar

[2] Shahriyar Amini, Jialiu Lin, Jason I Hong, Janne Lindqvist, and Joy Zhang. 2013. Mobile Application Evaluation Using Automation and Crowdsourcing. In Workshop on Privacy Enhancing Tools.Search in Google Scholar

[3] Karissa Bell. 2015. The 7 best iPhone photography apps of all time. (2015). http://mashable.com/2015/12/13/best-iphone-photo-apps-of-all-time/#NruakC5LTuqF.Search in Google Scholar

[4] K. Benton, L. J. Camp, and V. Garg. 2013. Studying the effectiveness of Android application permissions requests. In Security and Social Networking (SESOC), 2013 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops). 291–296.10.1109/PerComW.2013.6529497Search in Google Scholar

[5] Sven Bugiel, Lucas Davi, Alexandra Dmitrienko, Thomas Fischer, Ahmad-Reza Sadeghi, and Bhargava Shastry. 2012. Towards Taming Privilege-Escalation Attacks on Android. In 19th Network & Distributed System Security Symposium.Search in Google Scholar

[6] Iker Burguera, Urko Zurutuza, and Simin Nadjm-Tehrani. 2011. Crowdroid: behavior-based malware detection system for android. In Proceedings of the 1st ACM workshop on Security and privacy in smartphones and mobile devices. ACM, 15–26.Search in Google Scholar

[7] L. Jean Camp and Allan Friedman. 2004. Peer Patching – Rapid Response in Distributed Systems. In 24th Army Science Conference.Search in Google Scholar

[8] Scott Clifford, Ryan M Jewell, and Philip D Waggoner. 2015. Are samples drawn from Mechanical Turk valid for research on political ideology? Research & Politics 2, 4 (2015), 2053168015622072.Search in Google Scholar

[9] Ryszard Wiśniewski Connor Tumbleson. 2015. A tool for reverse engineering Android apk files. (2015). https://ibotpeaches.github.io/Apktool/.Search in Google Scholar

[10] E. Damiani, S. De Capitani Di Vimercati, S. Paraboschi, and P. Samarati. 2004. P2P-Based Collaborative Spam Detection and Filtering. In Fourth IEEE Conference on P2P. http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1334945Search in Google Scholar

[11] Developers. 2016. Requesting Permissions. (2016). https://developer.android.com/guide/topics/permissions/requesting.html.Search in Google Scholar

[12] Irit Dinur and Kobbi Nissim. 2003. Revealing information while preserving privacy. In Proceedings of the twenty-second ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems. ACM, 202–210.Search in Google Scholar

[13] Alex Dobie. 2015. The best photography apps for Android. (2015). http://www.androidcentral.com/best-photography-apps-android.Search in Google Scholar

[14] Zheng Dong and L Jean Camp. 2012. PeerSec: Towards Peer Production and Crowdsourcing for Enhanced Security.. In HotSec.Search in Google Scholar

[15] Zheng Dong, Vaibhav Garg, Jean Camp, and Apu Kapadia. 2012. Pools, Clubs and Security: Designing for a Party Not a Person. In Proceedings of The New Security Paradigms Workshop (NSPW). 77–86. DOI:http://dx.doi.org/10.1145/2413296.241330410.1145/2413296.2413304Open DOISearch in Google Scholar

[16] Paul D. Ellis. 2009. Thresholds for Interpreting Effect Sizes. (2009). http://www.polyu.edu.hk/mm/effectsizefaqs/thresholds_for_interpreting_effect_sizes2.html.Search in Google Scholar

[17] Kassem Fawaz and Kang G Shin. 2014. Location privacy protection for smartphone users. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security. ACM, 239–250.Search in Google Scholar

[18] Adrienne Porter Felt, Erika Chin, Steve Hanna, Dawn Song, and David Wagner. 2011a. Android permissions demystified. In 18th ACM Conference on Computer and Communications Security. 627–638. DOI:http://dx.doi.org/10.1145/2046707.204677910.1145/2046707.2046779Open DOISearch in Google Scholar

[19] Adrienne Porter Felt, Kate Greenwood, and David Wagner. 2011b. The effectiveness of application permissions. In 2nd USENIX Conference on Web Application Development. 75–86.Search in Google Scholar

[20] Adrienne Porter Felt, Elizabeth Ha, Serge Egelman, Ariel Haney, Erika Chin, and David Wagner. 2012. Android Permissions: User Attention, Comprehension, and Behavior. In 8th Symposium on Usable Privacy and Security. Article 3, 14 pages. DOI:http://dx.doi.org/10.1145/2335356.233536010.1145/2335356.2335360Search in Google Scholar

[21] Vaibhav Garg, Sameer Patil, Apu Kapadia, and L. Jean Camp. 2013. Peer-produced Privacy Protection. In IEEE International Symposium on Technology and Society (ISTAS). 147–154. DOI: http://dx.doi.org/10.1109/ISTAS.2013.661311410.1109/ISTAS.2013.6613114Open DOISearch in Google Scholar

[22] Hamza Harkous, Rameez Rahman, Bojan Karlas, and Karl Aberer. 2016. The Curious Case of the PDF Converter that Likes Mozart: Dissecting and Mitigating the Privacy Risk of Personal Cloud Apps. arXiv preprint arXiv:1608.05661 (2016).10.1515/popets-2016-0032Search in Google Scholar

[23] Jeff Howe. 2006. Crowdsourcing: A Definition. (2006). http://crowdsourcing.typepad.com/cs/2006/06/crowdsourcing_a.html.Search in Google Scholar

[24] Qatrunnada Ismail, Tousif Ahmed, Apu Kapadia, and Michael K Reiter. 2015. Crowdsourced exploration of security configurations. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 467–476.Search in Google Scholar

[25] ISO. 2016. Usability of consumer products and products for public use. (2016). https://www.iso.org/obp/ui/#iso:std:iso:ts:20282:-2:ed-2:v1:en.Search in Google Scholar

[26] Ruogu Kang, Stephanie Brown, Laura Dabbish, and Sara Kiesler. 2014. Privacy Attitudes of Mechanical Turk Workers and the U.S. Public. In Symposium On Usable Privacy and Security (SOUPS ’14). 37–49.Search in Google Scholar

[27] Patrick Gage Kelley, Sunny Consolvo, Lorrie Faith Cranor, Jaeyeon Jung, Norman Sadeh, and David Wetherall. 2012. A conundrum of permissions: Installing applications on an Android smartphone. In Financial Cryptography and Data Security. Springer, 68–79.Search in Google Scholar

[28] Patrick Gage Kelley, Lorrie Faith Cranor, and Norman Sadeh. 2013. Privacy as part of the app decision-making process. In 2013 ACM Conference on Human Factors in Computing Systems. 3393–3402.10.1145/2470654.2466466Search in Google Scholar

[29] Kristen Kennedy, Eric Gustafson, and Hao Chen. 2013. Quantifying the Effects of Removing Permissions from Android Applications. In IEEE Mobile Security Technologies.Search in Google Scholar

[30] Kim Komando. 2015. These 7 apps are among the worst at protecting privacy. (2015). http://www.usatoday.com/story/tech/columnist/komando/2015/09/18/apps-protecting-privacy/32563419/.Search in Google Scholar

[31] Robert Kosara and Caroline Ziemkiewicz. 2010. Do Mechanical Turks dream of square pie charts?. In Proceedings of the 3rd BELIV’10 Workshop: Beyond time and errors: novel evaluation methods for Information Visualization. ACM, 63–70.Search in Google Scholar

[32] Jialiu Lin, Bin Liu, Norman Sadeh, and Jason I Hong. 2014. Modeling users’ mobile app privacy preferences: Restoring usability in a sea of permission settings. In Symposium On Usable Privacy and Security (SOUPS 2014). 199–212.Search in Google Scholar

[33] Jialiu Lin, Norman Sadeh, Shahriyar Amini, Janne Lindqvist, Jason I Hong, and Joy Zhang. 2012. Expectation and purpose: Understanding users’ mental models of mobile app privacy through crowdsourcing. In 2012 ACM Conference on Ubiquitous Computing. 501–510.Search in Google Scholar

[34] Bin Liu, Mads Schaarup Andersen, Florian Schaub, Hazim Almuhimedi, Shikun Aerin Zhang, Norman Sadeh, Yuvraj Agarwal, and Alessandro Acquisti. 2016. Follow My Recommendations: A Personalized Assistant for Mobile App Permissions. In Twelfth Symposium on Usable Privacy and Security (SOUPS 2016).Search in Google Scholar

[35] Di Liu, Randolph G. Bias, Matthew Lease, and Rebecca Kuipers. 2012. Crowdsourcing for usability testing. Proceedings of the American Society for Information Science and Technology 49, 1 (2012), 1–10. DOI:http://dx.doi.org/10.1002/meet.1450490110010.1002/meet.14504901100Open DOISearch in Google Scholar

[36] Craig M. MacDonald, Sean Fitzell, Megan Koontz, Kate Merlie, Alana Miller, Samantha Raddatz, Tal Rozen, April Siqueiros, and Susan Young. 2013. Usability Report. (2013). http://www.craigmacdonald.com/wp-content/uploads/2013/08/CCPS-Usability-Report-Summer-2013.pdf.Search in Google Scholar

[37] Michelle L Mazurek, Saranga Komanduri, Timothy Vidas, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor, Patrick Gage Kelley, Richard Shay, and Blase Ur. 2013. Measuring password guessability for an entire university. In Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security. ACM, 173–186.Search in Google Scholar

[38] Eyal Peer, Joachim Vosgerau, and Alessandro Acquisti. 2014. Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods 46, 4 (2014), 1023–1031.10.3758/s13428-013-0434-y24356996Open DOISearch in Google Scholar

[39] PrivacyGrade. 2014. PrivacyGrade: Grading The Privacy Of Smartphone Apps. (2014). http://www.privacygrade.org/.Search in Google Scholar

[40] John Patrick Pullen. 2015. Your Favorite Apps Know More About You Than You Realize. (2015). http://time.com/3857380/apps-security-privacy-trivia-crack/.Search in Google Scholar

[41] Bahman Rashidi, Carol Fung, and Tam Vu. 2015. Dude, ask the experts!: Android resource access permission recommendation with RecDroid. In Integrated Network Management (IM), 2015 IFIP/IEEE International Symposium on. 296–304. DOI:http://dx.doi.org/10.1109/INM.2015.714030410.1109/INM.2015.7140304Open DOISearch in Google Scholar

[42] M. K. Reiter and S. G. Stubblebine. 1998. Resilient authentication using path independence. IEEE Trans. Comput. 47, 12 (1998), 1351–1362.10.1109/12.737682Open DOISearch in Google Scholar

[43] Meredith Rizzo. 2014. How Well Do Your Apps Protect Your Privacy? (2014). http://www.npr.org/sections/health-shots/2014/11/20/363342736/how-well-do-your-apps-protect-your-privacy.Search in Google Scholar

[44] Franziska Roesner, Tadayoshi Kohno, Alexander Moshchuk, Bryan Parno, Helen J Wang, and Crispin Cowan. 2012. Userdriven access control: Rethinking permission granting in modern operating systems. In Security and privacy (SP), 2012 IEEE Symposium on. IEEE, 224–238.Search in Google Scholar

[45] Joel Ross, Lilly Irani, M. Six Silberman, Andrew Zaldivar, and Bill Tomlinson. 2010. Who Are the Crowdworkers?: Shifting Demographics in Mechanical Turk. In CHI ’10 Extended Abstracts on Human Factors in Computing Systems (CHI EA ’10). ACM, New York, NY, USA, 2863–2872.Search in Google Scholar

[46] Jeff Sauro. 2010. If You Could Only Ask One Question, Use This One. (2010). http://www.measuringu.com/blog/single-question.php.Search in Google Scholar

[47] Jeff Sauro. 2011. Measuring Usability With The System Usability Scale (SUS). (2011). http://www.measuringu.com/sus.php.Search in Google Scholar

[48] Jeff Sauro. 2012. 10 Things To Know About The Single Ease Question (SEQ). (2012). http://www.measuringu.com/blog/seq10.php.Search in Google Scholar

[49] Jeff Sauro and Joseph S Dumas. 2009. Comparison of three one-question, post-task usability questionnaires. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1599–1608.Search in Google Scholar

[50] Umesh Shankar and Chris Karlof. 2006. Doppelganger: Better browser privacy without the bother. In Proceedings of the 13th ACM conference on Computer and communications security. ACM, 154–167.Search in Google Scholar

[51] Statista. 2016. Most popular global mobile messenger apps as of January 2016, based on number of monthly active users (in millions). (2016). http://www.statista.com/statistics/258749/most-popular-global-mobile-messenger-apps/.Search in Google Scholar

[52] Statista. 2017a. Number of monthly active Facebook Messenger users from April 2014 to April 2017 (in millions). (2017). https://www.statista.com/statistics/417295/facebook-messenger-monthly-active-users/.Search in Google Scholar

[53] Statista. 2017b. Number of monthly active Instagram users from January 2013 to April 2017 (in millions). (2017). https://www.statista.com/statistics/253577/number-of-monthly-active-instagram-users/.Search in Google Scholar

[54] Statista. 2017c. Number of monthly active Twitter users worldwide from 1st quarter 2010 to 1st quarter 2017 (in millions). (2017). https://www.statista.com/statistics/282087/number-of-monthly-active-twitter-users/.Search in Google Scholar

[55] Eran Toch. 2014. Crowdsourcing privacy preferences in context-aware applications. Personal and ubiquitous computing 18, 1 (2014), 129–141.Search in Google Scholar

[56] Voidcan. 2015. Top 10 Microblogging Sites for 2015. (2015). http://www.voidcan.org/top-10-microblogging-sites-for-2015/.Search in Google Scholar

[57] Haoyu Wang, Jason Hong, and Yao Guo. 2015. Using text mining to infer the purpose of permission use in mobile apps. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 1107–1118.Search in Google Scholar

[58] D. Wendlandt, D. G. Andersen, and A. Perrig. 2008. Perspectives: Improving SSH-style host authentication with multi-path probing. In Proceedings of USENIX Annual Technical Conference.Search in Google Scholar

[59] Primal Wijesekera, Arjun Baokar, Ashkan Hosseini, Serge Egelman, David Wagner, and Konstantin Beznosov. 2015. Android permissions remystified: a field study on contextual integrity. In 24th USENIX Security Symposium (USENIX Security 15). 499–514.Search in Google Scholar

[60] Wiki. 2016. Fair Payment. (2016). http://wiki.wearedynamo.org/index.php?title=Fair_payment.Search in Google Scholar

[61] Rebecca N Wright, L Jean Camp, Ian Goldberg, Ronald L Rivest, and Graham Wood. 2002. Privacy tradeoffs: myth or reality?. In Financial Cryptography. Springer, 147–151.Search in Google Scholar

[62] M. Yuen, I. King, and K. Leung. 2011. A Survey of Crowdsourcing Systems. In in Proceedings the 3rd SocialCom. http://www.cse.cuhk.edu.hk/~king/PUB/SocialCom2011-Yuen.pdf10.1109/PASSAT/SocialCom.2011.203Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo