Privacy-Preserving AI: Enabling Data Science on Unseen Data with PySyft and Differential Privacy

This podcast episode features Andrew Trask, creator of OpenMined, discussing the critical need and emerging solutions for privacy-preserving AI. The core problem addressed is the inaccessibility of sensitive private data, such as medical records, which forces the machine learning community to focus on less impactful, publicly available datasets. Trask argues that this limitation prevents AI from tackling some of society's most pressing issues like dementia or diabetes, despite the existence of powerful deep learning tools. The overarching goal of his work with OpenMined and tools like PySyft is to lower the barrier to entry for privacy-preserving machine learning, making it as easy to access and utilize private datasets as it is to `pip install` a public one.

Trask introduces several key techniques to achieve this. The first is **Remote Execution**, exemplified by PySyft, which extends deep learning frameworks like PyTorch. This allows data scientists to send their computational models to a remote "worker" (e.g., a hospital data center) where the data resides, execute operations, and receive pointers to results, all without the raw data ever leaving its secure location. The second technique is **Private Search and Sampling**, facilitated by PyGrid, which enables data scientists to search for relevant datasets and receive rich metadata (descriptions, ranges, types) and even small, curated samples, allowing for feature engineering and quality evaluation without direct access to sensitive information.

The third and most crucial technique discussed is **Differential Privacy**. Trask explains its intuition as a method for performing statistical analysis while guaranteeing the privacy of individual records. This is achieved by adding carefully calibrated noise to query results, ensuring that the output does not significantly change if any single individual's data is removed or altered. He uses the "randomized response" technique from political science as an analogy, where noise (coin flips) provides plausible deniability to survey participants. Trask strongly critiques traditional data anonymization, citing the de-anonymization of the Netflix Prize dataset and medical records as examples of its inherent dangers due to statistical uniqueness. Differential privacy, with its "epsilon" privacy budget, offers a formal, rigorous mechanism to quantify and control the amount of statistical uniqueness released, aiming to learn generalizable patterns (e.g., causes of cancer) rather than individual specifics.

While these tools address the challenge of data privacy, Trask acknowledges remaining hurdles. These include the risk to valuable AI models when sent to remote machines, the complexity of performing computations across multiple untrusting data owners, and the fundamental question of how to verify that remote computations are executed as intended. He briefly introduces Secure Multi-Party Computation as a potential solution for some of these issues, highlighting the ongoing research and development needed to realize a future where individuals can even set their own personal privacy budgets, thereby democratizing access to and control over their data in the age of AI.

Is it possible to answer questions using data that we cannot see?

The reason is it getting access to private data data about people was really really hard and as a result we spend most of our time working on problems and tasks like this.

Why is it that we spend all our time on tests like this when these tasks these represent you know our our friends loved ones and problems in society that really really matter?

The idea here is to lower the barrier to entry to increase the accessibility to some of the most important problems that we would like to address.

Differential privacy loosely stated is a field that it allows you to do statistical analysis without compromising the privacy of the data set.

This concept of adding noise to data to give plausible deniability is whether the secret weapon of differential privacy.

By and large it does not work. You don't remember anything else from this talk is very dangerous to do just data set anonymization.

The true answer of who should be setting epsilon budgets although logistically it's gonna be challenging... it should be us, it should be people in it should be people around their own information.