1. bookVolume 2022 (2022): Issue 1 (January 2022)
Journal Details
First Published
16 Apr 2015
Publication timeframe
4 times per year
access type Open Access

Ulixes: Facial Recognition Privacy with Adversarial Machine Learning

Published Online: 20 Nov 2021
Page range: 148 - 165
Received: 31 May 2021
Accepted: 16 Sep 2021
Journal Details
First Published
16 Apr 2015
Publication timeframe
4 times per year

Facial recognition tools are becoming exceptionally accurate in identifying people from images. However, this comes at the cost of privacy for users of online services with photo management (e.g. social media platforms). Particularly troubling is the ability to leverage unsupervised learning to recognize faces even when the user has not labeled their images. In this paper we propose Ulixes, a strategy to generate visually non-invasive facial noise masks that yield adversarial examples, preventing the formation of identifiable user clusters in the embedding space of facial encoders. This is applicable even when a user is unmasked and labeled images are available online. We demonstrate the effectiveness of Ulixes by showing that various classification and clustering methods cannot reliably label the adversarial examples we generate. We also study the effects of Ulixes in various black-box settings and compare it to the current state of the art in adversarial machine learning. Finally, we challenge the effectiveness of Ulixes against adversarially trained models and show that it is robust to countermeasures.


[1] European Parliament and Council of European Union, “Regulation (eu) 2016/679,” 2016, http://data.europa.eu/eli/reg/2016/679/oj. Search in Google Scholar

[2] “Are Organizations Ready for New Privacy Regulations?” [Online]. Available: https://www.internetsociety.org/resources/ota/2019/are-organizations-ready-for-new-privacy-regulations/ Search in Google Scholar

[3] J. P. Pesce, D. L. Casas, G. Rauber, and V. Almeida, “Privacy attacks in social media using photo tagging networks: a case study with Facebook,” in Proceedings of the 1st Workshop on Privacy and Security in Online Social Media - PSOSM ’12. Lyon, France: ACM Press, 2012, pp. 1–8. [Online]. Available: http://dl.acm.org/citation.cfm?doid=2185354.218535810.1145/2185354.2185358 Search in Google Scholar

[4] S. Shan, E. Wenger, J. Zhang, H. Li, H. Zheng, and B. Y. Zhao, “Fawkes: Protecting privacy against unauthorized deep learning models,” in 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, Aug. 2020, pp. 1589–1604. [Online]. Available: https://www.usenix.org/conference/usenixsecurity20/presentation/shan Search in Google Scholar

[5] C. Gao, V. Chandrasekaran, K. Fawaz, and S. Jha, “Face-off: Adversarial face obfuscation,” 2020.10.2478/popets-2021-0032 Search in Google Scholar

[6] F. Schroff, D. Kalenichenko, and J. Philbin, “FaceNet: A unified embedding for face recognition and clustering,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, Jun. 2015, pp. 815–823. [Online]. Available: http://ieeexplore.ieee.org/document/7298682/10.1109/CVPR.2015.7298682 Search in Google Scholar

[7] Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “DeepFace: Closing the Gap to Human-Level Performance in Face Verification,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, Jun. 2014, pp. 1701–1708. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=690961610.1109/CVPR.2014.220 Search in Google Scholar

[8] ——, “Web-scale training for face identification,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, Jun. 2015, pp. 2746–2754. [Online]. Available: http://ieeexplore.ieee.org/document/7298891/ Search in Google Scholar

[9] Y. Sun, Y. Chen, X. Wang, and X. Tang, “Deep learning face representation by joint identificationverification,” in Advances in Neural Information Processing Systems 27, Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2014, pp. 1988–1996. [Online]. Available: http://papers.nips.cc/paper/5416-deep-learning-face-representation-by-joint-identification-verification.pdf Search in Google Scholar

[10] Y. Sun, X. Wang, and X. Tang, “Deep Learning Face Representation from Predicting 10,000 Classes,” in 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, Jun. 2014, pp. 1891–1898. [Online]. Available: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=690964010.1109/CVPR.2014.244 Search in Google Scholar

[11] ——, “Deeply learned face representations are sparse, selective, and robust,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, Jun. 2015, pp. 2892–2900. [Online]. Available: http://ieeexplore.ieee.org/document/7298907/ Search in Google Scholar

[12] Y. Sun, D. Liang, X. Wang, and X. Tang, “Deepid3: Face recognition with very deep neural networks,” arXiv preprint arXiv:1502.00873, vol. abs/1502.00873, 2015. Search in Google Scholar

[13] L. Huang, A. D. Joseph, B. Nelson, B. I. Rubinstein, and J. D. Tygar, “Adversarial machine learning,” in Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence, ser. AISec ’11. New York, NY, USA: Association for Computing Machinery, 2011, p. 43–58. [Online]. Available: https://doi.org/10.1145/2046684.204669210.1145/2046684.2046692 Search in Google Scholar

[14] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” 2014. Search in Google Scholar

[15] S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, Jun. 2016, pp. 2574–2582. [Online]. Available: http://ieeexplore.ieee.org/document/7780651/10.1109/CVPR.2016.282 Search in Google Scholar

[16] A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial machine learning at scale,” 2016. Search in Google Scholar

[17] ——, “Adversarial examples in the physical world,” ICLR Workshop, 2017. [Online]. Available: https://arxiv.org/abs/1607.02533 Search in Google Scholar

[18] R. Awasthi, “Breaking Deep Learning with Adversarial examples using Tensorflow,” May 2018. [Online]. Available: https://cv-tricks.com/how-to/breaking-deep-learning-with-adversarial-examples-using-tensorflow/ Search in Google Scholar

[19] Y. Liu, X. Chen, C. Liu, and D. Song, “Delving into transferable adversarial examples and black-box attacks,” CoRR, vol. abs/1611.02770, 2016. [Online]. Available: http://arxiv.org/abs/1611.02770 Search in Google Scholar

[20] I. Evtimov, P. Sturmfels, and T. Kohno, “Foggysight: A scheme for facial lookup privacy,” CoRR, vol. abs/2012.08588, 2020. [Online]. Available: https://arxiv.org/abs/2012.08588 Search in Google Scholar

[21] V. Cherepanova, M. Goldblum, H. Foley, S. Duan, J. P. Dickerson, G. Taylor, and T. Goldstein, “Lowkey: Leveraging adversarial attacks to protect social media users from facial recognition,” in International Conference on Learning Representations, 2021. [Online]. Available: https://openreview.net/forum?id=hJmtwocEqzc Search in Google Scholar

[22] E. Radiya-Dixit and F. Tramèr, “Data poisoning won’t save you from facial recognition,” 2021. Search in Google Scholar

[23] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image Quality Assessment: From Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, Apr. 2004. [Online]. Available: http://ieeexplore.ieee.org/document/1284395/10.1109/TIP.2003.819861 Search in Google Scholar

[24] H.-W. Ng and S. Winkler, “A data-driven approach to cleaning large face datasets,” in 2014 IEEE International Conference on Image Processing (ICIP). Paris, France: IEEE, Oct. 2014, pp. 343–347. [Online]. Available: http://ieeexplore.ieee.org/document/7025068/ Search in Google Scholar

[25] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao, “Joint face detection and alignment using multitask cascaded convolutional networks,” IEEE Signal Processing Letters, vol. 23, no. 10, pp. 1499–1503, 2016. Search in Google Scholar

[26] B. Amos, B. Ludwiczuk, and M. Satyanarayanan, “Openface: A general-purpose face recognition library with mobile applications,” CMU-CS-16-118, CMU School of Computer Science, Tech. Rep., 2016. Search in Google Scholar

[27] J. Liu, Y. Deng, T. Bai, and C. Huang, “Targeting ultimate accuracy: Face recognition via deep embedding,” CoRR, vol. abs/1506.07310, 2015. [Online]. Available: http://arxiv.org/abs/1506.07310 Search in Google Scholar

[28] C. Ding and D. Tao, “Robust face recognition via multimodal deep face representation,” IEEE Transactions on Multimedia, vol. 17, no. 11, pp. 2049–2058, 2015. Search in Google Scholar

[29] S. Sankaranarayanan, A. Alavi, and R. Chellappa, “Triplet similarity embedding for face verification,” CoRR, vol. abs/1602.03418, 2016. [Online]. Available: http://arxiv.org/abs/1602.03418 Search in Google Scholar

[30] S. Sankaranarayanan, A. Alavi, C. D. Castillo, and R. Chellappa, “Triplet probabilistic embedding for face verification and clustering,” in 2016 IEEE 8th International Conference on Biometrics Theory, Applications and Systems (BTAS), 2016, pp. 1–8.10.1109/BTAS.2016.7791205 Search in Google Scholar

[31] X. Zhao, X. Liang, C. Zhao, M. Tang, and J. Wang, “Real-Time Multi-Scale Face Detector on Embedded Devices,” Sensors (Basel, Switzerland), vol. 19, no. 9, May 2019. [Online]. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6539187/10.3390/s19092158 Search in Google Scholar

[32] L. v. d. Maaten and G. Hinton, “Visualizing Data using t-SNE,” Journal of Machine Learning Research, vol. 9, no. Nov, pp. 2579–2605, 2008. [Online]. Available: https://www.jmlr.org/papers/v9/vandermaaten08a.html Search in Google Scholar

[33] G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in unconstrained environments,” University of Massachusetts, Amherst, Tech. Rep. 07-49, October 2007. Search in Google Scholar

[34] N. Pinto, Z. Stone, T. Zickler, and D. Cox, “Scaling up biologically-inspired computer vision: A case study in unconstrained face recognition on facebook,” in CVPR 2011 WORKSHOPS, Jun. 2011, pp. 35–42, iSSN: 2160-7516. Search in Google Scholar

[35] Systems Incorporated, Adobe, “Adobe photoshop lightroom classic cc help,” Tech. Rep., Feb. 2018. [Online]. Available: https://helpx.adobe.com/pdf/lightroom_reference.pdf Search in Google Scholar

[36] “Face Searching - Face++ Cognitive Services.” [Online]. Available: https://www.faceplusplus.com/face-searching/ Search in Google Scholar

[37] N. Papernot, P. D. McDaniel, and I. J. Goodfellow, “Transferability in machine learning: from phenomena to black-box attacks using adversarial samples,” CoRR, vol. abs/1605.07277, 2016. [Online]. Available: http://arxiv.org/abs/1605.07277 Search in Google Scholar

[38] D. Yi, Z. Lei, S. Liao, and S. Z. Li, “Learning face representation from scratch,” CoRR, vol. abs/1411.7923, 2014. [Online]. Available: http://arxiv.org/abs/1411.7923 Search in Google Scholar

[39] I. William, D. R. Ignatius Moses Setiadi, E. H. Rachmawanto, H. A. Santoso, and C. A. Sari, “Face recognition using facenet (survey, performance test, and comparison),” in 2019 Fourth International Conference on Informatics and Computing (ICIC), 2019, pp. 1–6.10.1109/ICIC47613.2019.8985786 Search in Google Scholar

[40] Q. Cao, L. Shen, W. Xie, O. M. Parkhi, and A. Zisserman, “Vggface2: A dataset for recognising faces across pose and age,” CoRR, vol. abs/1710.08092, 2017. [Online]. Available: http://arxiv.org/abs/1710.08092 Search in Google Scholar

[41] O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in Proceedings of the British Machine Vision Conference (BMVC), X. Xie, M. W. Jones, and G. K. L. Tam, Eds. BMVA Press, September 2015, pp. 41.1–41.12. [Online]. Available: https://dx.doi.org/10.5244/C.29.4110.5244/C.29.41 Search in Google Scholar

[42] A. Athalye, N. Carlini, and D. Wagner, “Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples,” in Proceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., vol. 80. Stockholmsmässan, Stockholm Sweden: PMLR, 10–15 Jul 2018, pp. 274–283. [Online]. Available: http://proceedings.mlr.press/v80/athalye18a.html Search in Google Scholar

[43] U. Shaham, Y. Yamada, and S. Negahban, “Understanding adversarial training: Increasing local stability of supervised models through robust optimization,” Neurocomputing, vol. 307, pp. 195–204, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S092523121830455710.1016/j.neucom.2018.04.027 Search in Google Scholar

[44] A. Paudice, L. Muñoz-González, A. Gyorgy, and E. C. Lupu, “Detection of adversarial training examples in poisoning attacks through anomaly detection,” 2018. Search in Google Scholar

[45] B. Wang, Y. Yao, S. Shan, H. Li, B. Viswanath, H. Zheng, and B. Y. Zhao, “Neural cleanse: Identifying and mitigating back-door attacks in neural networks,” in 2019 IEEE Symposium on Security and Privacy (SP), 2019, pp. 707–723.10.1109/SP.2019.00031 Search in Google Scholar

[46] E. Zhou, Z. Cao, and Q. Yin, “Naive-deep face recognition: Touching the limit of LFW benchmark or not?” CoRR, vol. abs/1501.04690, 2015. [Online]. Available: http://arxiv.org/abs/1501.04690 Search in Google Scholar

[47] N. Crosswhite, J. Byrne, C. Stauffer, O. Parkhi, Q. Cao, and A. Zisserman, “Template adaptation for face verification and identification,” Image and Vision Computing, vol. 79, pp. 35–48, 2018. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S026288561830147110.1016/j.imavis.2018.09.002 Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo