1. bookVolume 2019 (2019): Issue 2 (April 2019)
Journal Details
License
Format
Journal
eISSN
2299-0984
First Published
16 Apr 2015
Publication timeframe
4 times per year
Languages
English
access type Open Access

Privacy Protection for Audio Sensing Against Multi-Microphone Adversaries

Published Online: 04 May 2019
Volume & Issue: Volume 2019 (2019) - Issue 2 (April 2019)
Page range: 146 - 165
Received: 31 Aug 2018
Accepted: 16 Dec 2018
Journal Details
License
Format
Journal
eISSN
2299-0984
First Published
16 Apr 2015
Publication timeframe
4 times per year
Languages
English
Abstract

Audio-based sensing enables fine-grained human activity detection, such as sensing hand gestures and contact-free estimation of the breathing rate. A passive adversary, equipped with microphones, can leverage the ongoing sensing to infer private information about individuals. Further, with multiple microphones, a beamforming-capable adversary can defeat the previously-proposed privacy protection obfuscation techniques. Such an adversary can isolate the obfuscation signal and cancel it, even when situated behind a wall. AudioSentry is the first to address the privacy problem in audio sensing by protecting the users against a multi-microphone adversary. It utilizes the commodity and audio-capable devices, already available in the user’s environment, to form a distributed obfuscator array. AudioSentry packs a novel technique to carefully generate obfuscation beams in different directions, preventing the multi-microphone adversary from canceling the obfuscation signal. AudioSentry follows by a dynamic channel estimation scheme to preserve authorized sensing under obfuscation. AudioSentry offers the advantages of being practical to deploy and effective against an adversary with a large number of microphones. Our extensive evaluations with commodity devices show that protects the user’s privacy against a 16-microphone adversary with only four commodity obfuscators, regardless of the adversary’s position. AudioSentry provides its privacy-preserving features with little overhead on the authorized sensor.

Keywords

[1] [n. d.]. Amazon Alexa Premium Far-Field Voice Development Kit. https://developer.amazon.com/alexa-voice-service/devkits/amazon-premium-voice. ([n. d.]).Search in Google Scholar

[2] [n. d.]. Apple Ipad Tech Specs. https://www.apple.com/ipad-9.7/specs/. ([n. d.]).Search in Google Scholar

[3] [n. d.]. HomePod reinvents music in the home. https://www.apple.com/newsroom/2017/06/homepod-reinventsmusic-in-the-home/. ([n. d.]).Search in Google Scholar

[4] [n. d.]. Pixel 2 Tech Specs. https://store.google.com/us/product/pixel_2_specs?hl=en-US. ([n. d.]).Search in Google Scholar

[5] 2017. The Nielsen Total Audience Report: Q2 2017. (November 2017). Retrieved May 8, 2018 from http://www.nielsen.com/us/en/insights/reports/2017/thenielsen-total-audience-q2-2017.htmlSearch in Google Scholar

[6] 2018. Elliptic Labs Makes Smart Speakers More Intelligent with New INNER REFLECTION™ Ultrasound Virtual Sensor. (February 2018). Retrieved May 8, 2018 from http://www.ellipticlabs.com/2018/02/26/elliptic-labstouch-free-ultrasound-gesture-technology-leveraging\-thequalcomm-snapdragon-neural-processing-engine-2/Search in Google Scholar

[7] 2018. The Smart Audio Report, Fall/Winter 2017 from NPR and Edison Research. (January 2018). Retrieved May 8, 2018 from https://www.nationalpublicmedia.com/smart-audioreport/Search in Google Scholar

[8] Md Tanvir Islam Aumi, Sidhant Gupta, Mayank Goel, Eric Larson, and Shwetak Patel. 2013. DopLink: using the doppler effect for multi-device interaction. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. ACM, 583–586.Search in Google Scholar

[9] Daniel S. Berger, Francesco Gringoli, Nicolò Facchi, Ivan Martinovic, and Jens Schmitt. 2014. Gaining Insight on Friendly Jamming in a Real-world IEEE 802.11 Network. In Proceedings of the 2014 ACM Conference on Security and Privacy in Wireless & Mobile Networks (WiSec ’14). 105–116. https://doi.org/10.1145/2627393.262740310.1145/2627393.2627403Search in Google Scholar

[10] Andreas Braun, Stefan Krepp, and Arjan Kuijper. 2015. Acoustic tracking of hand activities on surfaces. In Proceedings of the 2nd international Workshop on Sensor-based Activity Recognition and Interaction. ACM, 9.Search in Google Scholar

[11] James Brown, Ibrahim Ethem Bagci, Alex King, and Utz Roedig. 2013. Defend Your Home#: Jamming Unsolicited Messages in the Smart Home. In Proceedings of the 2Nd ACM Workshop on Hot Topics on Wireless Network Security and Privacy (HotWiSec ’13). 1–6. https://doi.org/10.1145/2463183.246318510.1145/2463183.2463185Open DOISearch in Google Scholar

[12] Ke-Yu Chen, Daniel Ashbrook, Mayank Goel, Sung-Hyuck Lee, and Shwetak Patel. 2014. AirLink: sharing files between multiple devices using in-air gestures. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 565–569.Search in Google Scholar

[13] Qi Alfred Chen, Zhiyun Qian, and Zhuoqing Morley Mao. 2014. Peeking into Your App without Actually Seeing It: UI State Inference and Novel Android Attacks.. In USENIX Security Symposium. 1037–1052.Search in Google Scholar

[14] Seungeun Chung and Injong Rhee. 2016. vTrack: virtual trackpad interface using mm-level sound source localization for mobile interaction. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. ACM, 41–44.Search in Google Scholar

[15] Amit Das, Ivan Tashev, and Shoaib Mohammed. 2017. Ultrasound based gesture recognition. In Acoustics, Speech and Signal Processing (ICASSP), 2017 IEEE International Conference on. IEEE, 406–410.Search in Google Scholar

[16] Lucas Davi, Alexandra Dmitrienko, Ahmad-Reza Sadeghi, and Marcel Winandy. 2010. Privilege escalation attacks on android. In international conference on Information security. Springer, 346–360.Search in Google Scholar

[17] William Enck, Peter Gilbert, Seungyeop Han, Vasant Tendulkar, Byung-Gon Chun, Landon P Cox, Jaeyeon Jung, Patrick McDaniel, and Anmol N Sheth. 2014. TaintDroid: an information-flow tracking system for realtime privacy monitoring on smartphones. ACM Transactions on Computer Systems (TOCS) 32, 2 (2014), 5.Search in Google Scholar

[18] Viktor Erdélyi, Trung-Kien Le, Bobby Bhattacharjee, Peter Druschel, and Nobutaka Ono. 2018. Sonoloc: Scalable positioning of commodity mobile devices. (2018).10.1145/3210240.3210324Search in Google Scholar

[19] Angelo Farina. 2007. Advancements in impulse response measurements by sine sweeps. In Audio Engineering Society Convention 122. Audio Engineering Society.Search in Google Scholar

[20] Kassem Fawaz, Kyu-Han Kim, and Kang G Shin. 2016. Protecting Privacy of BLE Device Users.. In USENIX Security Symposium. 1205–1221.Search in Google Scholar

[21] Biying Fu, Dinesh Vaithyalingam Gangatharan, Arjan Kuijper, Florian Kirchbuchner, and Andreas Braun. 2017. Exercise Monitoring On Consumer Smart Phones Using Ultrasonic Sensing. In Proceedings of the 4th international Workshop on Sensor-based Activity Recognition and Interaction. ACM, 9.Search in Google Scholar

[22] Biying Fu, Jakob Karolus, Tobias Grosse-Puppendahl, Jonathan Hermann, and Arjan Kuijper. 2015. Opportunities for activity recognition using ultrasound doppler sensing on unmodified mobile phones. In Proceedings of the 2nd international Workshop on Sensor-based Activity Recognition and Interaction. ACM, 8.Search in Google Scholar

[23] Shyamnath Gollakota, Haitham Hassanieh, Benjamin Ransford, Dina Katabi, and Kevin Fu. 2011. They can hear your heartbeats: non-invasive security for implantable medical devices. In ACM SIGCOMM Computer Communication Review, Vol. 41. ACM, 2–13.Search in Google Scholar

[24] Sidhant Gupta, Daniel Morris, Shwetak Patel, and Desney Tan. 2012. Soundwave: using the doppler effect to sense gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1911–1914.Search in Google Scholar

[25] Chris Harrison, Julia Schwarz, and Scott E Hudson. 2011. TapSense: enhancing finger interaction on touch surfaces. In Proceedings of the 24th annual ACM symposium on User interface software and technology. ACM, 627–636.Search in Google Scholar

[26] Wenchao Huang, Yan Xiong, Xiang-Yang Li, Hao Lin, Xufei Mao, Panlong Yang, and Yunhao Liu. 2014. Shake and walk: Acoustic direction finding and fine-grained indoor localization using smartphones. In INFOCOM, 2014 Proceedings IEEE. IEEE, 370–378.Search in Google Scholar

[27] Yu Seung Kim and Patrick Tague. 2014. Proximity-based Wireless Access Control Through Considerate Jamming. In Proceedings of the ACM MobiCom Workshop on Security and Privacy in Mobile Environments (SPME ’14). 19–24. https://doi.org/10.1145/2646584.264658810.1145/2646584.2646588Open DOISearch in Google Scholar

[28] Tao Li, Yimin Chen, Jingchao Sun, Xiaocong Jin, and Yanchao Zhang. 2016. iLock: Immediate and Automatic Locking of Mobile Devices Against Data Theft. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS ’16). ACM, New York, NY, USA, 933–944. https://doi.org/10.1145/2976749.297829410.1145/2976749.2978294Open DOISearch in Google Scholar

[29] Jian Liu, Yan Wang, Gorkem Kar, Yingying Chen, Jie Yang, and Marco Gruteser. 2015. Snooping keystrokes with mmlevel audio ranging on a single phone. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking. ACM, 142–154.Search in Google Scholar

[30] Wenguang Mao, Jian He, and Lili Qiu. 2016. CAT: high-precision acoustic motion tracking. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. ACM, 69–81.Search in Google Scholar

[31] Ivan Martinovic, Paul Pichota, and Jens B. Schmitt. 2009. Jamming for Good: A Fresh Approach to Authentic Communication in WSNs. In Proceedings of the Second ACM Conference on Wireless Network Security (WiSec ’09). 161–168. https://doi.org/10.1145/1514274.151429810.1145/1514274.1514298Search in Google Scholar

[32] Raghuraman Mudumbai, D Richard Brown Iii, Upamanyu Madhow, and H Vincent Poor. 2009. Distributed transmit beamforming: challenges and recent progress. IEEE Communications Magazine 47, 2 (2009), 102–110.Search in Google Scholar

[33] Rajalakshmi Nandakumar, Shyamnath Gollakota, and Nathaniel Watson. 2015. Contactless sleep apnea detection on smartphones. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 45–57.Search in Google Scholar

[34] Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. Fingerio: Using active sonar for fine-grained finger tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1515–1525.Search in Google Scholar

[35] Rajalakshmi Nandakumar, Alex Takakuwa, Tadayoshi Kohno, and Shyamnath Gollakota. 2017. CovertBand: Activity Information Leakage Using Music. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 87 (Sept. 2017), 24 pages. https://doi.org/10.1145/313189710.1145/3131897Open DOISearch in Google Scholar

[36] Chunyi Peng, Guobin Shen, Yongguang Zhang, Yanlin Li, and Kun Tan. 2007. Beepbeep: a high accuracy acoustic ranging system using cots mobile devices. In Proceedings of the 5th international conference on Embedded networked sensor systems. ACM, 1–14.Search in Google Scholar

[37] Kun Qian, Chenshu Wu, Fu Xiao, Yue Zheng, Yi Zhang, Zheng Yang, and Yunhao Liu. 2018. Acousticcardiogram: Monitoring Heartbeats using Acoustic Signals on Smart Devices. In Proceedings of the IEEE International Conference on Computer Communications (Infocom ’18).10.1109/INFOCOM.2018.8485978Search in Google Scholar

[38] Yue Qiao, Ouyang Zhang, Wenjie Zhou, Kannan Srinivasan, and Anish Arora. 2016. PhyCloak: Obfuscating Sensing from Communication Signals.. In USENIX Annual Technical Conference.Search in Google Scholar

[39] Hanif Rahbari and Marwan Krunz. 2014. Friendly CryptoJam: A Mechanism for Securing Physical-layer Attributes. In Proceedings of the 2014 ACM Conference on Security and Privacy in Wireless & Mobile Networks (WiSec ’14). 129–140. https://doi.org/10.1145/2627393.262741510.1145/2627393.2627415Open DOISearch in Google Scholar

[40] Nirupam Roy, Haitham Hassanieh, and Romit Roy Choudhury. 2017. Backdoor: Making microphones hear inaudible sounds. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 2–14.Search in Google Scholar

[41] Nirupam Roy, Sheng Shen, Haitham Hassanieh, and Romit Roy Choudhury. 2018. Inaudible Voice Commands: The Long-Range Attack and Defense. In 15th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 18). 547–560.Search in Google Scholar

[42] Wenjie Ruan, Quan Z Sheng, Lei Yang, Tao Gu, Peipei Xu, and Longfei Shangguan. 2016. AudioGest: enabling fine-grained hand gesture detection by decoding echo signal. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 474–485.Search in Google Scholar

[43] Roman Schlegel, Kehuan Zhang, Xiao-yong Zhou, Mehool Intwala, Apu Kapadia, and XiaoFeng Wang. 2011. Sound-comber: A Stealthy and Context-Aware Sound Trojan for Smartphones.. In NDSS, Vol. 11. 17–33.Search in Google Scholar

[44] Matthias Schulz, Francesco Gringoli, Daniel Steinmetzer, Michael Koch, and Matthias Hollick. 2017. Massive Reactive Smartphone-based Jamming Using Arbitrary Waveforms and Adaptive Power Control. In Proceedings of the 10th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec ’17). 111–121. https://doi.org/10.1145/3098243.309825310.1145/3098243.3098253Search in Google Scholar

[45] W. Shen, P. Ning, X. He, and H. Dai. 2013. Ally Friendly Jamming: How to Jam Your Enemy and Maintain Your Own Wireless Connectivity at the Same Time. In 2013 IEEE Symposium on Security and Privacy. 174–188. https://doi.org/10.1109/SP.2013.2210.1109/SP.2013.22Open DOISearch in Google Scholar

[46] Adam Smith, Hari Balakrishnan, Michel Goraczko, and Nissanka Priyantha. 2004. Tracking moving devices with the cricket location system. In Proceedings of the 2nd international conference on Mobile systems, applications, and services. ACM, 190–202.Search in Google Scholar

[47] Zheng Sun, Aveek Purohit, Raja Bose, and Pei Zhang. 2013. Spartacus: spatially-aware interaction for mobile devices through energy-efficient audio sensing. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services. ACM, 263–276.Search in Google Scholar

[48] Sanjib Sur, Teng Wei, and Xinyu Zhang. 2014. Autodirective audio capturing through a synchronized smartphone array. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. ACM, 28–41.Search in Google Scholar

[49] Nils Ole Tippenhauer, Luka Malisa, Aanjhan Ranganathan, and Srdjan Capkun. 2013. On limitations of friendly jamming for confidentiality. In Security and Privacy (SP), 2013 IEEE Symposium on. IEEE, 160–173.Search in Google Scholar

[50] Barry D Van Veen and Kevin M Buckley. 1988. Beamforming: A versatile approach to spatial filtering. IEEE assp magazine 5, 2 (1988), 4–24.Search in Google Scholar

[51] Junyi Wang, Zhou Lan, Chin-Sean Sum, Chang-Woo Pyo, Jing Gao, Tuncer Baykas, Azizur Rahman, Ryuhei Funada, Fumihide Kojima, Ismail Lakkis, et al. 2009. Beamforming codebook design and performance evaluation for 60GHz wideband WPANs. In Vehicular Technology Conference Fall (VTC 2009-Fall), 2009 IEEE 70th. IEEE, 1–6.Search in Google Scholar

[52] Jue Wang, Deepak Vasisht, and Dina Katabi. 2014. RFIDraw: virtual touch screen in the air using RF signals. In ACM SIGCOMM Computer Communication Review, Vol. 44. ACM, 235–246.Search in Google Scholar

[53] Tianben Wang, Daqing Zhang, Yuanqing Zheng, Tao Gu, Xingshe Zhou, and Bernadette Dorizzi. 2018. C-FMCW Based Contactless Respiration Detection Using Acoustic Signal. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 4, Article 170 (Jan. 2018), 20 pages. https://doi.org/10.1145/316118810.1145/3161188Open DOISearch in Google Scholar

[54] Wei Wang, Alex X Liu, and Ke Sun. 2016. Device-free gesture tracking using acoustic signals. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. ACM, 82–94.Search in Google Scholar

[55] Xuyu Wang, Runze Huang, and Shiwen Mao. 2017. Sonar-Beat: Sonar phase for breathing beat monitoring with smart-phones. In Computer Communication and Networks (ICCCN), 2017 26th International Conference on. IEEE, 1–8.Search in Google Scholar

[56] Hiroki Watanabe, Tsutomu Terada, and Masahiko Tsukamoto. 2013. Ultrasound-based movement sensing, gesture-, and context-recognition. In Proceedings of the 2013 International Symposium on Wearable Computers. ACM, 57–64.Search in Google Scholar

[57] Ryan West. 2008. The psychology of security. Commun. ACM 51, 4 (2008), 34–40.Search in Google Scholar

[58] Matthias Wilhelm, Ivan Martinovic, Jens B. Schmitt, and Vincent Lenders. 2011. WiFire: A Firewall for Wireless Networks. In Proceedings of the ACM SIGCOMM 2011 Conference (SIGCOMM ’11). 456–457. https://doi.org/10.1145/2018436.201851810.1145/2018436.2018518Open DOISearch in Google Scholar

[59] Sangki Yun, Yi-Chao Chen, and Lili Qiu. 2015. Turning a mobile device into a mouse in the air. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 15–29.Search in Google Scholar

[60] Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. 2017. Strata: Fine-Grained Acoustic-based Device-Free Tracking. In Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys ’17). 15–28. https://doi.org/10.1145/3081333.308135610.1145/3081333.3081356Open DOISearch in Google Scholar

[61] B. Zhang, Q. Zhan, S. Chen, M. Li, K. Ren, C. Wang, and D. Ma. 2014. ssrP riW hisper : Enabling Keyless Secure Acoustic Communication for Smartphones. IEEE Internet of Things Journal 1, 1 (Feb 2014), 33–45. https://doi.org/10.1109/JIOT.2014.229799810.1109/JIOT.2014.2297998Open DOISearch in Google Scholar

[62] Guoming Zhang, Chen Yan, Xiaoyu Ji, Tianchen Zhang, Taimin Zhang, and Wenyuan Xu. 2017. DolphinAttack: Inaudible voice commands. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. ACM, 103–117.Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo