Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Models. (arXiv:2305.15594v1 [cs.LG])

Large language models (LLMs) are excellent in-context learners. However, the
sensitivity of data contained in prompts raises privacy concerns. Our work
first shows that these concerns are valid: we instantiate a simple but highly
effective membership inference attack against the data used to prompt LLMs. To
address this vulnerability, one could forego prompting and resort to
fine-tuning LLMs with known algorithms for private gradient descent. However,
this comes at the expense of the practicality and efficiency offered by
prompting. Therefore, we propose to privately learn to prompt. We first show
that soft prompts can be obtained privately through gradient descent on
downstream data. However, this is not the case for discrete prompts. Thus, we
orchestrate a noisy vote among an ensemble of LLMs presented with different
prompts, i.e., a flock of stochastic parrots. The vote privately transfers the
flock’s knowledge into a single public prompt. We show that LLMs prompted with
our private algorithms closely match the non-private baselines. For example,
using GPT3 as the base model, we achieve a downstream accuracy of 92.7% on the
sst2 dataset with ($epsilon=0.147, delta=10^{-6}$)-differential privacy vs.
95.2% for the non-private baseline. Through our experiments, we also show that
our prompt-based approach is easily deployed with existing commercial APIs.