# Decoding generative AI’s privacy paradox **Author** Madelyn Goodman **Date**: September 7, 2023 **TLDR**: As generative AI tools continue to improve, developers need more data to train models with higher efficacy. This need for more data comes at odds with prioritizing [data privacy](/content/glossary/data-privacy/index.html) while at the same time giving us the power to safeguard sensitive data with self-deployed tools. Here we take a deep dive into the benefits and drawbacks of generative AI for data privacy and security. ## Generative AI and Work This time last year if you had never heard of GPT-3 or DALL-E 2 you wouldn’t have necessarily been at a disadvantage. Now it seems if you don’t have your ChatGPT tab open at all times you’re lagging behind. Generative AI increases the efficiency of work across industries by automating rote tasks, accelerating upskilling, and allowing for the quicker development of proofs of concept. The development and improvement of these models, however, comes at a cost. Every time we ask ChatGPT for help, we are exposing our data to an external source, risking our privacy. ## The Threat Generative AI Poses to Data Privacy The key ingredients for the recent breakthroughs in generative AI are model architectures, enormous training datasets, and massive computational resources. The importance of high-quality data for training these models means that foundation model providers are incentivized to draw on their customer data for training. Because of the potential of these models to emit sensitive information contained in their training data, everyone using generative AI is concerned about one thing: data privacy. Below is a comparison of the accuracy of several different LLMs based on their number of parameters and amount of training data represented by the number of tokens. The color gradient represents model performance, with a darker color representing higher performance. _Figure adapted from “_ [_Chinchilla’s Wild Implications_](https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla-s-wild-implications) _”_ This research from [Deep Mind](http://arxiv.org/pdf/2203.15556) shows that smaller models, trained on larger datasets, could outperform larger models with more parameters, such as GPT-3. Data is key for scaling model improvement. Smaller open-source models with under 100B parameters trained on increasingly large corpuses of text are becoming more and more common. Meta’s Llama 2 models, for example, were trained on datasets of 2 trillion tokens. Building the next generation of models will require even _more_ data, whether for pre-training or fine-tuning. This demand for data could become fundamentally at odds with privacy and security requirements. When using LLMs in your organization, it’s important to be cautious. Your team could be unintentionally sending your data to third parties with disparate data security policies from yours and therefore sharing confidential information in breach of regulations. For example, data submitted to OpenAI from 2020 through March 2023 was potentially used to train future models, such as ChatGPT. The risk of a model disclosing information from its training set is proportional to the size of the model (the number of parameters) and the number of times that particular data point showed up in the training set. Research by Carlini et al exposed severe training data leakage in diffusion models as shown below. _Figure from “_ [_Extracting Training Data from Diffusion Models_](https://arxiv.org/abs/2301.13188) _”_ Training data leakage is a significant liability for an organization. If sensitive data is used in training, it is at risk of being disclosed, plain and simple. Companies like Stability AI and Microsoft are currently facing the consequences, with lawsuits stemming from their models reproducing copyrighted content without proper credit or compensation to the original creators. ## How to Preserve Data Privacy with Generative AI All is not lost, however, when it comes to preserving data privacy while also reaping the benefits of generative AI. There are tools that allow you to use your data with third-party models like ChatGPT and Stability AI safely. Beyond these third-party hosted models, there are other options for integrating LLMs in your organization. ### Tools for using third-party hosted models safely Generative AI can be used to protect data when sending requests to third-party hosted models. LLMs can auto-redact information from a dataset. Recognizing the context of a sentence that can make a word sensitive was a limitation with previous auto-redaction methods. For example, take the following sentence: > _“The President tested positive for COVID-19 again Saturday per a letter from presidential physician Dr. Kevin O’Connor.”_ Previous techniques would only label “Saturday” and “Dr. Kevin O’Connor” as time and person words. However, “president” and “COVID-19” are sensitive aspects of the sentence as well. Transformer models can detect context, allowing for accurate labeling. Advancements in prompt engineering have also improved LLMs’ value in executing this task, with research from [Carnegie Mellon](https://arxiv.org/abs/2305.15444v2) showing how careful few-shot prompting can generate consistent and accurate results. ### Deploying models on premises Most major cloud providers are expected to offer ways to deploy large third-party models in your own VPC. Alternatively, training and hosting your own LLM on premises is becoming more accessible as smaller fine-tuned models perform better on some tasks than large general-purpose models. Like the [Chinchilla model](https://arxiv.org/abs/2203.15556), these models have fewer parameters but are trained on larger sets of more specialized data. These models perform comparably to their larger third-party hosted counterparts on common benchmarks. StabilityAI’s open-sourced model [Stable Beluga 2](https://huggingface.co/stabilityai/StableBeluga2) showed almost the same level of accuracy as the closed-source [GPT-4](https://openai.com/research/gpt-4), around 60%, on a benchmark called [TruthfulQA](https://arxiv.org/abs/2109.07958) that measures a model’s proneness to reproducing falsehoods found on the internet. Current research suggests that GPT-4 has been producing less accurate results, as the model is fine-tuned broadly; however, locally tuned models will not have this problem because you can control the fine-tuning yourself. ## How Generative AI Can Be Used to _Protect_ Data Privacy We are increasingly seeing a trend of leveraging large, general-purpose models to generate synthetic data, which is used to supplement real datasets to train more specialized models. Synthetic data is artificially generated to mimic real information without using actual data points, ensuring privacy and security. A group at [Stanford](https://scontent-sjc3-1.xx.fbcdn.net/v/t39.8562-6/333078981_693988129081760_4712707815225756708_n.pdf?_nc_cat=108&ccb=1-7&_nc_sid=ad8a9d&_nc_ohc=S5aoWeendYMAX_-Efk6&_nc_ht=scontent-sjc3-1.xx&oh=00_AfBqaSfpmW2dYgSgS_yfKnKdPvMuI5839z7tsd68-t3UVA&oe=642B5122) used the “[_self-instruct_](https://arxiv.org/abs/2212.10560)” method of model training, starting with a small seed of data from GPT-3 and Meta’s LLaMa model to generate a large amount of task-based data to fine-tune a ChatGPT-like model for just $700 as opposed to the millions it took to train ChatGPT. This model, trained on synthetic data, was not only more cost-effective but also more secure to use. Further research has been done to train models on “distilled” training data generated by larger models. Data is distilled from LLMs by following certain prompting patterns, often involving [chain of thought](https://arxiv.org/abs/2212.08410) tasks, to get more specific data for training smaller and more specialized models. These models show a [steeper scaling curve](http://arxiv.org/pdf/2301.12726) - demonstrating that the more specialized and distilled data LLMs are shown, the better they perform. Curious to learn more about the data privacy risks generative AI poses? Check out [this](/content/webinars/managing-the-data-risk-of-generative-ai/index.html) webinar from [Tonic.ai](/content/site-root.html) where founder and CEO, Ian Coe, and Head of AI, Ander Steele, discuss how to manage these risks. Also, tune into [our upcoming webinar, Data Safety in the Age of AI,](https://events.a.team/data-safety-in-the-age-of-ai) in partnership with [A.Team](https://www.a.team/) where Ander returns along with Anjana Harve, Seasoned Global Chief Digital & Information Officer, and Michael Rispin, General Counsel at Sprout Social, to discuss how to manage these risks. ## Further Resources 1. [Scaling Laws for Neural Language Models, Kaplan et al, 2020](http://arxiv.org/pdf/2001.08361) 2. [An empirical analysis of compute-optimal large language model training, Hoffman et al, 2022](https://arxiv.org/abs/2203.15556) 3. [Chinchilla’s Wild Implications, Nostalgebraist, 2022](https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla-s-wild-implications) 4. [GPT-4 Technical Report, OpenAI, 2023](https://cdn.openai.com/papers/gpt-4.pdf) 5. [Training Compute-Optimal Large Language Models, Hoffmann et al, 2022](https://arxiv.org/abs/2203.15556) 6. [Sparks of Artificial General Intelligence: Early experiments with GPT-4, Bubeck et al, 2023](http://arxiv.org/pdf/2303.12712) 7. [Extracting Training Data from Large Language Models, Carlini et al, 2021](http://arxiv.org/pdf/2012.07805) 8. [Extracting Training Data from Diffusion Models, Carlini et al, 2023](http://arxiv.org/pdf/2301.13188) 9. [LLaMA: Open and Efficient Foundation Language Models, Touvron, Lavril, Izacard et al, 2023](http://arxiv.org/pdf/2301.13188) 10. [PromptNER: Prompting for Named Entity Recognition, Ashok, Lipton, June 2023](http://arxiv.org/pdf/2305.15444v2) 11. [Alpaca: A Strong, Replicable Instruction-Following Model, Taori, Gulrajani, Zhang, Dubois, Li, et al, March 2023](https://crfm.stanford.edu/2023/03/13/alpaca.html) 12. [Self-Instruct: Aligning Language Models with Self-Generated Instructions, Wang, Kordi, Mishra, Liu, Smith, et al, May 2023](http://arxiv.org/pdf/2212.10560) 13. [Teaching Small Language Models to Reason, Magister, Mallinson, Adamek, Malmi, Severyn, June 2023](http://arxiv.org/pdf/2212.08410) 14. [Specializing Smaller Language Models towards Multi-Step Reasoning, Fu, Peng, Ou, Sabharwal, Khot, January 2023](http://arxiv.org/pdf/2301.12726)