Adversarial Machine Learning, Security, and Trustworthy AI

In the old days, hackers modify code. Now, hackers can modify DATA. Such an adversary can force machine learning to make mistakes. We study why this happens and how to defend against it. Our research ranges from test-time attacks, training data poisoning attacks to other subtle forms of adversarial attacks. This page contains our research on the theory, algorithms, and applications of adversarial learning, security, and trustworthy AI.

Talks

ADVERSARY-AWARE LEARNING TECHNIQUES AND TRENDS IN CYBERSECURITY (ALEC). AAAI Fall Symposium, October 18-19, 2018. Arlington, VA
Toward adversarial learning as control at 2nd AOR/IARPA Workshop on Adversarial Machine Learning. University of Maryland, May 2018
Machine Teaching and its Applications. Department of Computer Science, Duke University. 2018
Debugging the Machine Learning Pipeline at the Interpretable Machine Learning Symposium, NIPS 2017
Talk at ICML 2016 Workshop on Reliable Machine Learning in the Wild, New York: Machine Teaching and Security

Publications

Yiding Chen and Xiaojin Zhu. Optimal attack against autoregressive models by manipulating the environment. In The Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI), 2020.
[pdf | arXiv | matlab code]

Yuzhe Ma, Xuezhou Zhang, Wen Sun, and Xiaojin Zhu. Policy poisoning in batch reinforcement learning and control. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[pdf | poster | code | arXiv]

Xuanqing Liu, Si Si, Xiaojin Zhu, Yang Li, and Cho-Jui Hsieh. A unified framework for data poisoning attack to graph-based semi-supervised learning. In Advances in Neural Information Processing Systems (NeurIPS), 2019.

Ayon Sen, Xiaojin Zhu, Liam Marshall, Robert Nowak. Should Adversarial Attacks Use Pixel p-Norm?. arXiv:1906.02439. 2019.
[link | code and data ]

Yuzhe Ma, Xiaojin Zhu, Justin Hsu. Data Poisoning against Differentially-Private Learners: Attacks and Defenses. In The 28th International Joint Conference on Artificial Intelligence (IJCAI), 2019.
[arxiv]

Xuezhou Zhang, Xiaojin Zhu, and Laurent Lessard. Online Data Poisoning Attacks. arXiv:1903.01666. 2019.
[link]

Yiding Chen and Xiaojin Zhu. Optimal Adversarial Attack on Autoregressive Models. arXiv:1902.00202, 2019.
[link | matlab code]

Owen Levin, Zihang Meng, Vikas Singh, Xiaojin Zhu. Fooling Computer Vision into Inferring the Wrong Body Mass Index. arXiv:1905.06916, 2019.
[link]

Xiaojin Zhu. An optimal control view of adversarial machine learning. arXiv:1811.04422, 2018.
[link]

Kwang-Sung Jun, Lihong Li, Yuzhe Ma, and Xiaojin Zhu. Adversarial attacks on stochastic bandits. In Advances in Neural Information Processing Systems (NIPS), 2018.
[pdf]

Ayon Sen, Scott Alfeld, Xuezhou Zhang, Ara Vartanian, Yuzhe Ma, and Xiaojin Zhu. Training set camouflage. In Conference on Decision and Game Theory for Security (GameSec), 2018
[pdf]

Yuzhe Ma, Kwang-Sung Jun, Lihong Li, and Xiaojin Zhu. Data poisoning attacks in contextual bandits. In Conference on Decision and Game Theory for Security (GameSec), 2018
[arXiv]

Xuezhou Zhang, Xiaojin Zhu, and Stephen Wright. Training set debugging using trusted items. In The Thirty-Second AAAI Conference on Artificial Intelligence (AAAI), 2018
[pdf]

Scott Alfeld, Xiaojin Zhu, and Paul Barford. Explicit defense actions against test-set attacks. In The Thirty-First AAAI Conference on Artificial Intelligence (AAAI), 2017.
[pdf]

Scott Alfeld, Xiaojin Zhu, and Paul Barford. Data Poisoning Attacks against Autoregressive Models. In The Thirtieth AAAI Conference on Artificial Intelligence (AAAI), 2016.
[pdf]

Gabriel Cadamuro, Ran Gilad-Bachrach, and Xiaojin Zhu. Debugging machine learning models. In ICML Workshop on Reliable Machine Learning in the Wild, 2016.
Training data repair to ensure certain test items are correctly predicted. An application of machine teaching.
[pdf | extended abstract for CHI 2016 workshop on human centred machine learning]

Shike Mei and Xiaojin Zhu. The security of latent Dirichlet allocation. In The Eighteenth International Conference on Artificial Intelligence and Statistics (AISTATS), 2015.
How might an attacker poison the corpus to manipulate LDA topics? We answer this question via machine teaching.
[pdf]

Shike Mei and Xiaojin Zhu. Using Machine Teaching to Identify Optimal Training-Set Attacks on Machine Learners. In The Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI-15), 2015.
An application of machine teaching to identify the optimal training-set-attacks against a learning algorithm.
[pdf | poster ad | poster | Mendota ice data | Tech Report 1813]

Shike Mei and Xiaojin Zhu. Some Submodular Data-Poisoning Attacks on Machine Learners. Computer Science Tech Report 1822, University of Wisconsin-Madison, 2015.
[pdf]

In the media

Air Force-backed center to make machine learning more independent, predictable, secure University of Wisconsin News. By Sam Million-Weaver and Adrienne Nienow. May 29, 2018

Squashing the bugs in machine learning: Researchers make computer-trained models more trustworthy Computer Sciences Department News, UW-Madison. By Jennifer Smith. March 8, 2018

Back to Professor Zhu's home page