Provenance related papers

A collection of data provenance papers & talks. Papers are classified by research area.

Papers

• Provenance in middleware
  • H5Prov: I/O Performance Analysis of Science Applications Using HDF5 File-level Provenance (CUG'19)
  • Scientific data exchange: a schema for HDF5-based storage of raw and analyzed data (Journal of Synchrotron Radiation 2014)
  • A Generic Provenance Middleware for Queries, Updates, and Transactions (TaPP'14)
  • Study in Usefulness of Middleware-Only Provenance (eScience'14)
• Provenance in e-Science
  • Provenance in Collaborative in Silico Scientific Research: a Survey (SIGMOD record'20)
  • ProvONE+: A Provenance Model for Scientific Workflows (WISE'20)
  • Workflow Provenance in the Lifecycle of Scientific Machine Learning (arXiv, Sept. 2020)
  • Provenance in Collaborative in Silico Scientific Research: a Survey (SIGMOD'20)
  • Efficient Runtime Capture of Multi-workflow Data Using Provenance (eScience'19)
  • Provenance Data in the Machine Learning Lifecycle in Computational Science and Engineering (WORKS'19)
  • The role of machine learning in scientific workflows (Int. J. HPC, 2019)
  • Efficient Runtime Capture of Multiworkflow Data Using Provenance(eScience'19)
  • Blockchain Based Provenance Sharing of Scientific Workflows (Big Data'18)
  • Big Provenance Stream Processing for Data Intensive Computations (eScience'18)
  • DAC-MAN: Data Change Management for Scientific Datasets on HPC systems (SC'18)
  • Capturing and Querying Workflow Runtime Provenance with PROV: a Practical Approach (EDBT'13)
  • End-to-End eScience: Integrating Workflow, Query, Visualization, and Provenance at an Ocean Observatory (eScience'08)
  • Provenance and Scientific Workflows: Challenges and Opportunities (SIGMOD'08)
  • A survey of data provenance in e-science (SIGMOD'05)
  • Lineage Retrieval for Scientific Data Processing: A Survey (ACM Computing Surveys, Mar. 2005)
• Provenance in database
  • A Provenance Storage Method Based On Parallel Database (ICISCE'20)
  • Going Beyond Provenance: Explaining Query Answers with Pattern-based Counterbalances (SIGMOD'19)
  • Provenance and Probabilities in Relational Databases (SIGMOD record'19)
  • Interoperability for Provenance-aware Databases using PROV and JSON (TaPP'15)
  • Why and Where􏰀 Characterization of Data Provenance (ICDT'01)
• Provenance in scripts
  • Improving Data Scientist Efficiency with Provenance (ICSE'20)
  • noWorkflow: a Tool for Collecting, Analyzing, and Managing Provenance from Python Scripts (VLDB'17)
• Provenance in machine learning
  • LIMA: Fine-grained Lineage Tracing and Reuse in Machine Learning Systems (SIGMOD'21)
  • PrIU: A Provenance-Based Approach for Incrementally Updating Regression Models (SIGMOD'20)
  • The Best of Both Worlds: Challenges in Linking Provenance and Explainability in Distributed Machine Learning (ICDCS'19)
• Provenance query
  • Ariadne: Online Provenance for Big Graph Analytics (SIGMOD'19)
  • Zoom*UserViews: querying relevant provenance in workflow systems (VLDB'07)
• Provenance visualization
  • Characterizing Provenance in Visualization and Data Analysis: An Organizational Framework of Provenance Types and Purposes (TVCG'16)
  • Evaluation of filesystem provenance visualization tools (TVCG'13)
  • Using VisTrails and Provenance for Teaching Scientific Visualization (COMPUTER GRAPHICS forum 2011)
• Provenance optimization
  • Hypothetical Reasoning via Provenance Abstraction (SIGMOD'19)
  • ProTracer: Towards Practical Provenance Tracing by Alternating Between Logging and Tainting (NDSS'16)
  • LogGC: Garbage Collecting Audit Log (CCS'13)
  • Provenance analysis: Towards quality provenance (eScience'12)
• Provenance standard
  • The W3C PROV family of specifications for modelling provenance metadata (EDBT '13)
  • A Framework for Collecting Provenance in Data-Centric Scientific Workflows (ICWS'06)
• Provenance systems
  • Improve Data Scientist Efficiency with Provenance (ICSE'20)
  • Runtime Analysis of Whole-System Provenance (CCS'18)
  • Provenance Integration Requires Reconciliation (TaPP'11)
  • Taverna, reloaded (SSDBM’10)
  • Layering in Provenance Systems (ATC'09)
  • Provenance-Aware Storage Systems (ATC'06)
  • Trio: A System for Integrated Management of Data, Accuracy, and Lineage (CIDR'05)
• Multi area
  • Data provenance: What’s next? (SIGMOD Record'18)
  • A survey on provenance: What for? What form? What from? (VLDB.J'17)
  • A Primer on Provenance (ACMqueue'14)
  • The Foundations for Provenance on the Web (Foundations and Trends in Web Science 2010)
  • Special Issue on Data Provenance (Data Engineering Bulletin Issues, Dec. 2007)
    • Provenance in Databases: Past, Current, and Future
    • Provenance and Data Synchronization
    • Program Slicing and Data Provenance
    • Recording Provenance for SQL Queries and Updates
    • Issues in Building Practical Provenance Systems
    • Provenance in Scientific Workflow Systems
    • Copyright and Provenance: Some Practical Problems
• RDF storage & hardware acceleration
  • Fast and Concurrent RDF Queries using RDMA-assisted GPU Graph Exploration (ATC'18)
  • Sub-millisecond Stateful Stream Querying over Fast-evolving Linked Data (SOSP'17)
  • Fast and Concurrent RDF Queries with RDMA- Based Distributed Graph Exploration (OSDI'16)
• FAIR principles: maturity evaluations and implimentation considerations

  • FAIR Principles: Interpretations and Implementation Considerations (Data Intelligence, 2020)

    • W3C Ontology

  • Evaluating FAIR maturity through a scalable, automated, community-governed framework (Scientific data, Nature, 2019)

    • Web APIs

  • Results of an Analysis of Existing FAIR Assessment Tools (2019)

  • The FAIR Guiding Principles for scientific data management and stewardship (Scientific data, Nature, 2016)

Provenance & Reliability & Security

• ProvTalk: Towards Interpretable Multi-level Provenance Analysis in Networking Functions Virtualization (NFV) (NDSS'21)
• On Optimizing the Trade-off between Privacy and Utility in Data Provenance (SIGMOD'21)
• Lineage Stash: Fault Tolerance Off the Critical Path (SOSP'19)
• Towards Scalable Cluster Auditing through Grammatical Inference over Provenance Graphs (NDSS'18)
• Practical Whole-System Provenance Capture (SoCC'17)
• FRAPpuccino: Fault-detection through Runtime Analysis of Provenance (HotCloud'17)
• Lineage-driven Fault Injection (SIGMOD'15)
• Trustworthy Whole-System Provenance for the Linux Kernel (Security'15)
• Diagnosing missing events in distributed systems with negative provenance (SIGCOMM'14)
• Hi-Fi: collecting high-fidelity whole-system provenance (Annual Computer Security Applications Conference 2012)
• Secure network provenance (SOSP'11)
• Backtracking Intrusions (ACM Transactions on Computer Systems '2005)

Talks

• Leveraging HDF5 infrastructure by ADF to build an interoperable package & contextualized data in Pharma using semantic technology
• HUG presentation about VOL connectors
  • Presentation
  • Video
• Documenting Scientific Workflows: The Metadata, Provenance & Ontology Project
  • Link1
  • Link2
• DacMan/DVC

Other Userful Links

• Versioned HDF5 library
  • Introducing Versioned HDF5
  • Design of the Versioned HDF5 Library
  • Performance of the Versioned HDF5 Library
• PROV-Overview
  • ProvONE: A PROV Extension Data Model for Scientific Workflow Provenance
  • ProvONE: extending PROV to support the DataONE scientific community
  • Wf4Ever Research Object Model 1.0
• The Importance of Data Set Provenance for Science
• FAIR Principles
  • The FAIR Guiding Principles for scientific data management and stewardship
• Open Access Datasets
  • https://livingatlas.arcgis.com/esri_ngda_datasets/
  • https://datasetsearch.research.google.com
  • https://ieee-dataport.org
  • https://www.dataone.org/fair/
  • https://www.data.gov