Skip to content

[ECCV24] "Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning" by Chongyu Fan*, Jiancheng Liu*, Alfred Hero, Sijia Liu

License

Notifications You must be signed in to change notification settings

OPTML-Group/Unlearn-WorstCase

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning

preprint issues

Venue:ECCV 2024 License: MIT GitHub top language GitHub repo size GitHub stars

Teaser Figure
Figure 1: An overview of unlearning under worst-case forget set vs. random forget set.

Welcome to the official repository of the ECCV 2024 paper "Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning".

Abstract

The trustworthy machine learning (ML) community is increasingly recognizing the crucial need for models capable of selectively 'unlearning' data points after training. This leads to the problem of machine unlearning (MU), aiming to eliminate the influence of chosen data points on model performance, while still maintaining the model's utility post-unlearning. Despite various MU methods for data influence erasure, evaluations have largely focused on random data forgetting, ignoring the vital inquiry into which subset should be chosen to truly gauge the authenticity of unlearning performance. To tackle this issue, we introduce a new evaluative angle for MU from an adversarial viewpoint. We propose identifying the data subset that presents the most significant challenge for influence erasure, i.e., pinpointing the worst-case forget set. Utilizing a bi-level optimization principle, we amplify unlearning challenges at the upper optimization level to emulate worst-case scenarios, while simultaneously engaging in standard training and unlearning at the lower level, achieving a balance between data influence erasure and model utility. Our proposal offers a worst-case evaluation of MU's resilience and effectiveness. Through extensive experiments across different datasets (including CIFAR-10, 100, CelebA, Tiny ImageNet, and ImageNet) and models (including both image classifiers and generative models), we expose critical pros and cons in existing (approximate) unlearning strategies. Our results illuminate the complex challenges of MU in practice, guiding the future development of more accurate and robust unlearning algorithms.

Quick start

Using indices directly

  • The indices of the worst-case forget set are available here.
  • The format of the indices is a list, named as {dataset name}_{forgetting data amount}_forget/remain.
train_set = CIFAR10(data_dir, train=True, transform=train_transform, download=True)

# load the indices of the worst-case forget set
with open('cifar10_4500_forget.pkl', 'rb') as f:
    forget_idx = pickle.load(f)
forget_set = Subset(train_set, forget_idx)

Supplementary materials

For appendix in ECCV camera-ready version, please refer to pdfs/appendix.pdf.

Contributors

Cite this work

@misc{fan2024challenging,
      title={Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning}, 
      author={Chongyu Fan and Jiancheng Liu and Alfred Hero and Sijia Liu},
      year={2024},
      eprint={2403.07362},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}

About

[ECCV24] "Challenging Forgets: Unveiling the Worst-Case Forget Sets in Machine Unlearning" by Chongyu Fan*, Jiancheng Liu*, Alfred Hero, Sijia Liu

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published