Daniel D. Johnson

Education

University of Toronto  |  Toronto, ON, Canada
PhD, Computer Science
Sept 2021 - present
Advisors: David Duvenaud, Chris J. Maddison
Harvey Mudd College  |  Claremont, CA  |  GPA: 3.98
Bachelor of Science, Joint Program in Computer Science and Mathematics
Aug 2014 - May 2018
Graduated with High Distinction and Honors in Computer Science, Mathematics, and Humanities

Research Interests

  • Understanding what neural network models know, how they represent and manipulate their knowledge, and how that knowledge generalizes to new situations.
  • Shaping model generalization behavior by changing their training data.
  • Ensuring language models and AI systems behave safely and beneficially, especially in the presence of uncertainty or computational constraints.
  • Teaching AI agents to honestly and accurately report their knowledge and capabilities.
  • Designing objectives that incentivise desired behavior across model scales and levels of competence.

Employment History

Research Scientist
Google DeepMind  |  Toronto, ON, Canada
Apr 2022 - present
Conducted research on probabilistic machine learning and uncertainty quantification applied to software development and language models. Developed open-source libraries for extracting and visualizing language model activations (Penzai and Treescope) and for summarizing uncertainty in code completion systems (R-U-SURE).
Research Software Engineer
Google Research, Brain team  |  Toronto, ON, Canada
June 2021 - Apr 2022
Conducted research on probabilistic machine learning and representation learning. Prototyped methods for summarizing language model uncertainty for code completion and developer assistance tasks.
AI Resident
Google Research, Brain team  |  Montréal, QC, Canada
Oct 2019 - June 2021
Conducted research on machine learning for software static analysis, diffusion models in discrete state spaces, and programming language design.
Software Engineer  |  Perception
Cruise Automation  |  San Francisco, CA, USA
Full time: July 2018 - Sept 2019. Intern: May 2017 - Aug 2017
Developed machine learning models for self-driving cars.

Conference and Journal Publications - Machine Learning

Experts Don't Cheat: Learning What You Don't Know By Predicting Pairs
Daniel D. Johnson, Daniel Tarlow, David Duvenaud, Chris J. Maddison
ICML 2024 [arXiv]
R-U-SURE? Uncertainty-Aware Code Suggestions By Maximizing Utility Across Random User Intents
Daniel D. Johnson, Daniel Tarlow, Christian Walder
ICML 2023 [arXiv, code]
A Density Estimation Perspective on Learning From Pairwise Human Preferences
Vincent Dumoulin, Daniel D. Johnson, Pablo Samuel Castro, Hugo Larochelle, Yann Dauphin
Transactions on Machine Learning Research (2023) [arXiv]
Contrastive Learning Can Find An Optimal Basis For Approximately View-Invariant Functions
Daniel D. Johnson, Ayoub El Hanchi, Chris J. Maddison
ICLR 2023 [arXiv]
Learning Generalized Gumbel-Max Causal Mechanisms
Guy Lorberbom*, Daniel D. Johnson*, Chris J. Maddison, Daniel Tarlow, Tamir Hazan
NeurIPS 2021 (spotlight) [arXiv, code]
Structured Denoising Diffusion Models In Discrete State-Spaces
Jacob Austin*, Daniel D. Johnson*, Jonathan Ho, Daniel Tarlow, Rianne van den Berg
NeurIPS 2021 [arXiv, code]
Learning Graph Structure With a Finite-State Automaton Layer
Daniel D. Johnson, Hugo Larochelle, Daniel Tarlow
NeurIPS 2020 (spotlight presentation); also presented at GRL+ 2020 [arXiv, talk, code]
Latent Gaussian Activity Propagation: Using Smoothness And Structure to Separate And Localize Sounds in Large Noisy Environments
Daniel D. Johnson, Daniel Gorelik, Ross E. Mawhorter, Kyle Suver, Weiqing Gu, Steven Xing, Cody Gabriel, Peter Sankhagowit
NeurIPS 2018 [pdf, poster]
Learning Graphical State Transitions
Daniel D. Johnson
ICLR 2017 (oral presentation) [pdf, talk, code, blogpost]
Learning to Create Jazz Melodies Using a Product of Experts
Daniel D. Johnson, Robert M. Keller, Nicholas Weintraut
International Conference on Computational Creativity 2017 [pdf, blogpost]
Generating Polyphonic Music With Tied-Parallel Networks
Daniel D. Johnson
EvoMusArt 2017 [pdf, code, blogpost]

Other Publications - Machine Learning

Penzai + Treescope: A Toolkit for Interpreting, Visualizing, and Editing Models As Data
Daniel D. Johnson
ICML Workshop on Mechanistic Interpretability 2024 [arXiv, Penzai code, Treescope code]
Uncertain Simulators Don't Always Simulate Uncertain Agents
Daniel D. Johnson
Technical blog post, 2023 [post]
A Library for Representing Python Programs as Graphs for Machine Learning
David Bieber, Kensen Shi, Petros Maniatis, Charles Sutton, Vincent Hellendoorn, Daniel Johnson, Daniel Tarlow
arXiv preprint [arXiv]
Beyond In-Place Corruption: Insertion And Deletion In Denoising Probabilistic Models
Daniel D. Johnson, Jacob Austin, Rianne van den Berg, Daniel Tarlow
2021 ICML Workshop on Invertible Neural Networks, Normalizing Flows, and Explicit Likelihood Models [arXiv, code]

Conference and Journal Publications - Programming and Mathematics

Parallel Algebraic Effect Handlers
Ningning Xie*, Daniel D. Johnson*, Dougal Maclaurin, Adam Paszke
International Conference on Functional Programming 2024 [paper, code]
Getting to the Point. Index Sets And Parallelism-Preserving Autodiff for Pointful Array Programming
Adam Paszke, Daniel D. Johnson, David Duvenaud, Dimitrios Vytiniotis, Alexey Radul, Matthew Johnson, Jonathan Ragan-Kelley, Dougal Maclaurin
International Conference on Functional Programming 2021 [arXiv, code]
Geometric Realizations of the 3D Associahedron (multimedia exposition)
Satyan L. Devadoss, Daniel D. Johnson, Justin Lee, Jackson Warley
International Symposium on Computational Geometry 2017 [pdf, demo]

Invited Talks

Experts Don’t Cheat: Learning What You Don’t Know By Predicting Pairs
ELLIS Robust LLMs Workshop – Keble College, Oxford, UK – July 2024

Honors and Awards

  • TMLR Expert Reviewer (2023, 2024)
  • NeurIPS Top 10% Reviewer (2020)
  • Computing Research Association Outstanding Undergraduate Researcher - Runner-up (2018)
  • Greever Clinic Award (Senior Capstone Project) (2018)
  • Barry Goldwater Scholar (2017)
  • Stavros Busenberg Prize for Outstanding Promise in Applied Mathematics (2017)
  • Robert James Prize for Outstanding Performance in Mathematics (2015)
  • Harvey S. Mudd Merit Award (2014-2018)
  • National Merit Scholar (2014)