Ankur Taly

Senior Staff Research Scientist, Google Inc. (2012 -- 2019 and Jul. 2020 onwards)

Adjunct Lecturer, Stanford Univeristy (Fall 2024, Fall 2023)

Computer Scientist interested in all forms of analysis: models, data, programs, protocols. Currently, focussed on data and model quality for Gemini models at Google.

Previously: Head of Data Science, Fiddler Labs (2019 -- 2020)

Ph.D. Stanford University; B. Tech IIT Bombay.

Email: ataly AT google.com
Curriculum Vitae: PDF (may be outdated)

News

[Sep. 2024] Co-teaching CS 329T:Trustworthy Machine Learning at Stanford University during Fall 2024.
[Aug. 2024] Co-instructed a tutorial on LLM grounding and evaluation at KDD 2024
[Jul. 2024] Joined the Gemini team to focus on post-training data and model quality for Gemini LLMs.
[Sep. 2022] Talk at National Academies Workshop on AI Confidence
[Sep. 2022] Paper on training data influence for text models at NeurIPS 2022
[Apr. 2022] Talk on Evaluating, Interpreting and Monitoring ML models at ODSC East 2022
[May. 2021] Paper on automated exploration of What-If scenarios at UAI 2021
[Aug. 2020] Best paper award at CD-MAKE 2020 for our Explanation Game paper (co-authored with Luke Merrick)!
[Jul. 2020] Back to doing research at Google!
[Jan. 2020] Co-instructed a tutorial on Explainable AI in industry at FAT* 2020. Slides, Video
[Aug. 2019] Co-instructed a tutorial on Explainable AI in industry at KDD 2019. Slides
[May. 2019] Started a new job at Fiddler labs!
[Feb. 2019] DREAM seminar on Integrated Gradients at UC Berkeley. Slides, Video
[Dec. 2018] Paper studying the impact of Integrated Gradient explanations on doctors grading Diabetic Retinopathy cases (in computer-assisted settings), accepted by the American Academy of Opthalmology. In popular press: here and here
[Jun. 2018] Paper on debugging question-answering networks accepted at ACL 2018
[Jun. 2017] Talk at SRI International on understanding deep neural networks. Slides
[May. 2017] Attribution paper accepted at ICML 2017.
[Mar. 2017] Our post on the Google DataScience Blog on attributing predictions made by deep neural networks.
[Oct. 2016] Outstanding paper award at ESORICS 2016 for our private discover paper
[Aug. 2016] Taught a short course on "Distributed Authorization" at FOSAD 2016 (summer school), Bertinoro, Italy

Research

Fiddler labs (2019 -- 2020)

Drove explainability techniques from research to industrial practice. Contributions: Contrastive explanations based on Shapley Values (paper), Conceptualized a model analysis framework called Slice and Explain, An attribution based method for obtaining algorithmic recourse.

Google LLC (2012 -- 2019)

Explaining Deep Neural Networks (2016 -- 2019): We study the problem of attributing the prediction of a deep network to its input features, as an attempt towards explaining individual predictions. For instance, in an object recognition network, an attribution method could tell us which pixels of the image were responsible for a certain label being picked (see this paper, this blog post, and these slides)

Distributed Authorization (2012 -- 2016): This work revolves around designing and analyzing identity, authentication, and authorization frameworks for the Web, Cloud and other distributed system settings. (see the "Macaroons" paper at NDSS 2014)

Stanford University (2007 -- 2012)

Semantics for JavaScript
- A small-step style operational semantics for SES_light subset of ES5-strict based JavaScript Click Here (based on 5th edition of ECMA-262 spec)
- A small-step style operational semantics for ES3 based JavaScript Click Here (based on 3rd edition of ECMA-262 spec)

Javascript Sandboxing (with Prof. John C. Mitchell and Dr. Sergio Maffeis): This work involved designing provably-secure language-based mechanisms for sandboxing untrusted JavaScript

Hybrid Systems Verification (with Dr. Ashish Tiwari): This work involved formulating deductive techniques for synthesis and verification of hybrid systems.

Teaching

CS 329T: Trustworthy Machine Learning at Stanford University
Graduate course, co-taught with Anupam Datta and John C. Mitchell, focussed on how to make ML models and applications (specifically LLMs) trustworthy and reliable
Offered in: Fall 2024, Fall 2023

Tutorials
- LLM Grounding and Evaluation
  Co-instructed with Krishnaram Kenthapady, Mehrnoosh Sameki
  Presented at: KDD 2024
- Explainable AI in Industry
  Co-instructed with Krishnaram Kenthapady,Sahin Geyik, Varun Mithal, and Krishna Gade
  Presented at: KDD 2019, FAT* 2020, and WWW 2020
Short course on "Practical Distributed Authorization" at FOSAD 2016 (summer school)
Lectures 1 and 2: PDF
Lectures 3 and 4: PDF

Guest Lectures
- Axiomatic Attribution for Deep Networks
  Interpretability and Explainability in Machine Learning (COMPSCI 282BR), Harvard University, Winter 2021
- The Explanation Game: Explaining Machine Learning Models using Shapley Values Slides
  Security and Fairness of Deep Learning (ECE739), CMU Silicon Valley, Winter 2020
- Explaining Machine Learning Models
  Engineering Safe Software Systems (ECE737), CMU Silicon Valley, Fall 2019
- Axiomatic Attribution for Deep Networks
  Security and Fairness of Deep Learning (ECE739), CMU Silicon Valley, Winter 2018
- Sandboxing Untrusted JavaScript
  Advanced Computer Security (CS223), University of California at Santa Cruz, 2012
- An Operational Semantics for JavaScript
  Programming Language Theory (CS258), Stanford University, 2010
Educational Videos

Students / Interns Advised

[2024] Nitish Joshi (PhD student at NYU): Worked on a novel and cheap method for fact verification.
[2023] Andrew Bai (PhD student at UCLA, co-advised with Chih-Kuan Yeh): Working on applications of training data influence to active learning.
[2022] Minghen Li (PhD student at UC Riverside, co-advised with Binbin Xiong): Worked on label noise detection in training sets.
[2021] Chih-kuan Yeh (PhD student at CMU): Worked on training data influence for NLP models.
[2020] Susan Hao (PhD student at UC Berkeley): Worked on automatically slicing error distributions of machine learning models.
[2018] Sahaj Garg (undergrad at Stanford University, jointly advised with Vincent Perot): Worked on enforcing fairness in text classification models.
[2017] Pramod Kaushik Mudrakarta (PhD student at University of Chicago): Worked on analyzing a deep network for question-answering---called Neural Programmer---in order to understand its input-output behavior, extract rules from it, and leverage weaknesses in its logic to construct attacks against it.
[2015] Siddhartha Jayanti (PhD student at MIT): Worked on formally verifying the Vanadium authorization protocol; see this
[2014] Andres Erbsen (PhD student at MIT): Worked on the Vanadium security model; see this

PC Participation

FAccT 2022, PLDI (ERC) 2019, PLDI (ERC) 2014, HOTSPOT 2013, PLAS 2013, POST 2014

Publications

Machine Learning, Natural Language Processing

Which Pretrain Samples to Rehearse when Finetuning Pretrained Models? -- Andrew Bai, Chih-Kuan Yeh, Cho-Jui Hsieh, Ankur Taly
In submission

Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey) -- Krishnaram Kenthapady, Mehrnoosh Sameki, Ankur Taly
Proc. of KDD 2024 PDF

Identifying and Mitigating the Security Risks of Generative AI -- Clark Barrett, Brad Boyd, Elie Burzstein, Nicholas Carlini, Brad Chen, Jihye Choi, Amrita Roy Chowdhury, Mihai Christodorescu, Anupam Datta, Soheil Feizi, Kathleen Fisher, Tatsunori Hashimoto, Dan Hendrycks, Somesh Jha, Daniel Kang, Florian Kerschbaum, Eric Mitchell, John Mitchell, Zulfikar Ramzan, Khawaja Shams, Dawn Song, Ankur Taly, Diyi Yang
Proc. of Foundations and Trends® in Privacy and Security 2023 PDF

Interpretable Mixture of Experts -- Aya Abdelsalam Ismail, Sercan Ö. Arik, Jinsung Yoon, Ankur Taly, Soheil Feizi, Tomas Pfister
Proc. of TMLR 2023 PDF

First is better than Last for Language Data Influence -- Chih-Kuan Yeh, Ankur Taly, Mukund Sundararajan, Frederick Liu, Pradeep Ravikumar
Proc. of NeurIPS 2022 PDF

A Multistakeholder Approach Towards Evaluating AI Transparency Mechanisms -- Ana Lucic, Madhulika Srikumar, Umang Bhatt, Alice Xiang, Ankur Taly, Q. Vera Liao, Maarten de Rijke
Proc. of CHI 2021 Workshop on Operationalizing Human-Centered Perspectives in Explainable AI PDF

Local Explanations via Necessity and Sufficiency: Unifying Theory and Practice -- David Watson, Limor Gultchin, Ankur Taly, Luciano Floridi
Proc. of Uncertainty in Artificial Intelligence (UAI) 2021 PDF

Explainable Machine Learning in Deployment -- Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yunhan Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, Peter Eckersley
Proc. of Fairness, Accountability, and Transparency (FAT*) 2020 PDF

The Explanation Game: Explaining Machine Learning Models with Cooperative Game Theory -- Luke Merrick, Ankur Taly
Proc. of Cross Domain conference on Machine Learning and Knowledge Extraction (CD-MAKE), 2020 PDF [Award paper]

Property Inference for Deep Neural Networks -- Divya Gopinath, Hayes Converse, Corina S. Pasareanu, Ankur Taly
Proc. of Automated Software Engineering (ASE) 2019 PDF

Using Attribution to Decode Dataset Bias in Neural Network Models for Chemistry -- Kevin McCloskey, Ankur Taly, Federico Monti, Michael P. Brenner, Lucy Colwell
Proc. of National Academy of Science (PNAS) 2019 PDF

Exploring Principled Visualizations of Deep Network Attributions - Mukund Sundararajan, Jinhua Xu, Ankur Taly, Rory Sayres, Amir Najmi
Proc. of IUI Workshop on Explainable Smart Systems (ExSS) 2019 PDF

Using a Deep Learning Algorithm and Integrated Gradients Explanation to Assist Grading for Diabetic Retinopathy - Rory Sayres, Ankur Taly et al.
Proc. of American Academy of Opthalmology 2018 PDF

Counterfactual Fairness in Text Classification through Robustness -- Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Alex Beutel, Ed H. Chi
Proc. of AIES 2018 PDF

Did the model understand the question? -- Pramod Mudrakarta, Ankur Taly, Mukund Sundararajan, Kedar Dhamdhere
Proc. of ACL 2018 PDF

It was the training data pruning too! -- Pramod Mudrakarta, Ankur Taly, Mukund Sundararajan, Kedar Dhamdhere
On arxiv, 2018 PDF

Assisted reads for diabetic retinopathy using a deep learning algorithm and integrated gradient explanation (extended abstract) -- Rory Sayres et al.
2018 Annual meeting of the Association for Research in Vision and Ophtalmology (ARVO), 2018 PDF

Abductive Matching in Question Answering -- Kedar Dhamdhere, Kevin Mccurley, Mukund Sundararajan, Ankur Taly
On arxiv, 2017 PDF

Axiomatic Attribution of Deep Networks -- Mukund Sundararajan, Ankur Taly, Qiqi Yan
Proc. of ICML 2017 PDF

Gradients of Counterfactuals -- Mukund Sundararajan, Ankur Taly, Qiqi Yan
On arxiv, 2016 PDF

Distributed Authorization

Distributed Authorization in Vanadium -- Andres Erbsen, Asim Shankar, and Ankur Taly
Book chapter in FOSAD VIII (lecture notes) PDF

Privacy, Discovery, and Authentication for the Internet of Things -- David Wu, Ankur Taly, Asim Shankar, Dan Boneh
Proc. of ESORICS 2016 PDF [Award paper]

Distributed Authorization with Distributed Grammars -- Martın Abadi, Mike Burrows, Himabindu Pucha, Adam Sadovsky, Asim Shankar, and Ankur Taly
Proc. of PLABS 2015 PDF

Macaroons: Cookies with Contextual Caveats for Decentralized Authorization in the Cloud -- Arnar Birgisson, Joe Politz, Ulfar Erlingsson, Ankur Taly, Michael Vrable, Mark Lentczner
Proc. of NDSS 2014 PDF

Language-based Security, Web Security

Automated Analysis of Security-critical JavaScript APIs -- Ankur Taly, Ulfar Erlingsson, Mark S. Miller, John C. Mitchell, Jasvir Nagra
Proc. of IEEE S&P (Oakland conference) 2011 PDF [Award paper]

Separation Logic and Mashup Isolation Problem -- Ankur Taly
Stanford University Tech Report, 2010 PDF

Object Capabilities and Isolation of Untrusted Web Applications -- Sergio Maffeis, John C. Mitchell, Ankur Taly
Proc. of IEEE S&P (Oakland conference) 2010 PDF

Isolating JavaScript with Filters, Rewriting, and Wrapper -- Sergio Maffeis, John C. Mitchell, Ankur Taly
Proc. of ESORICS 2009 PDF

Run-Time Enforcement of Secure JavaScript Subsets -- Sergio Maffeis, John C. Mitchell, Ankur Taly
W2SP 2009 PDF

Language based Isolation of Untrusted JavaScript -- Sergio Maffeis, Ankur Taly
Proc. of CSF 2009 PDF

An Operational Semantics for JavaScript -- Sergio Maffeis, John C. Mitchell, Ankur Taly
Proc. of APLAS 2008 PDF

Static Analysis, Verification, Synthesis

Automated Synthesis of Symbolic Instruction Encodings from I/O samples -- Patrice Godefroid, Ankur Taly
Proc. of PLDI 2012 PDF

Synthesizing Switching Logic using Constraint Solving -- Ankur Taly, Ashish Tiwari, Sumit Gulwani
Journal on Software Tools for Technology Transfer (STTT 2011) Springer Link

Switching Logic Synthesis for Reachability -- Ankur Taly, Ashish Tiwari
Proc. of EMSOFT 2010 PDF

Deductive Verification of Continuous Dynamical Systems -- Ankur Taly, Ashish Tiwari
Proc. of FSTTCS 2009 PDF

Synthesizing Switching Logic using Constraint Solving -- Ankur Taly, Ashish Tiwari, Sumit Gulwani
Proc. of VMCAI 2009 PDF

Static Analysis by Policy Iteration on Relational domains -- Stephane Gaubert, Eric Goubault, Ankur Taly, Sarah Zennou
Proc. of ESOP 2007 PDF

Efficient Symbolic Reachability of Networks of Transition Systems -- Sudeep Juvekar, Ankur Taly, Varun Kanade, Supratik Chakraborty
Proc. of General Motors Workshop on Next Generation Design and Verification Methodologies for Distributed Embedded Control Systems, Bangalore (2007) PDF

Blog posts

Monitoring Feature Attributions (co-authored with Claudiu Garcia and Kaz Sato), Sep. 2021
Counterfactual Explanations vs. Attribution based Explanations (co-authored with Aalok Shanbhag), Mar. 2020
Three Trends in Explainability, Feb. 2020
Causality in model explanations and in the real world (co-authored with Dan Frankowski), Jul. 2019
Attributing a deep network’s prediction to its input features, Mar. 2017

Dissertation

Sandboxing Untrusted JavaScript (Abstract, PDF, Slides)
PhD Dissertation, Stanford University, 2013