Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add UnavailableDataset as a default dataset for lite mode #2083

Merged
merged 15 commits into from
Sep 10, 2024

Conversation

ravi-kumar-pilla
Copy link
Contributor

@ravi-kumar-pilla ravi-kumar-pilla commented Sep 5, 2024

Description

Related to #2058

Development notes

  • Created a custom dataset UnavailableDataset
  • Updated the default dataset from MemoryDataset to UnavailableDataset in --lite mode.
  • Updated filepath to have an empty string as N/A instead of - as suggested by @stephkaiser
  • Updated tests

QA notes

  • When you run kedro viz --lite with kedro-datasets missing in the conda env, the metadata panel for the dataset should show the dataset type as UnavailableDataset if the dataset is mentioned in catalog.py (i.e., not a MemoryDataset)
Screenshot 2024-09-05 at 2 43 30 PM
  • All tests should pass

Checklist

  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added new entries to the RELEASE.md file
  • Added tests to cover my changes

@ravi-kumar-pilla ravi-kumar-pilla changed the title Chore/lite metatdata change Add UnavailableDataset as a default dataset for lite mode Sep 5, 2024
@ravi-kumar-pilla ravi-kumar-pilla marked this pull request as ready for review September 5, 2024 21:05
@ravi-kumar-pilla ravi-kumar-pilla requested review from stephkaiser and removed request for astrojuanlu September 5, 2024 21:05
@astrojuanlu
Copy link
Member

Thanks @ravi-kumar-pilla - could you clarify what was the situation before this PR?

@ravi-kumar-pilla
Copy link
Contributor Author

Thanks @ravi-kumar-pilla - could you clarify what was the situation before this PR?

Hi @astrojuanlu , Thanks for reviewing. Before this change, the default was MemoryDataset . This PR is part of making users aware of the missing dependency. MemoryDataset will be a bit misleading as the dataset is registered in the catalog.

@stephkaiser
Copy link

thanks Ravi! using Gitpod, filepath is showing as N/A for me 👍 but still shows MemoryDataset, maybe i did something wrong?

Screenshot 2024-09-10 at 12 14 13 Screenshot 2024-09-10 at 12 13 15

@ravi-kumar-pilla
Copy link
Contributor Author

thanks Ravi! using Gitpod, filepath is showing as N/A for me 👍 but still shows MemoryDataset, maybe i did something wrong?

Screenshot 2024-09-10 at 12 14 13 Screenshot 2024-09-10 at 12 13 15

Thanks for reviewing. Yes these are actually MemoryDatasets (as they are not registered in the DataCatalog). We will show UnavailableDataset to the datasets which are registered in the DataCatalog but the dependency is not installed.

For example, companies is registered in DataCatalog as below but has a missing dependency of pandas.CSVDataset, so companies metadata will have dataset type as UnavailableDataset. I hope this is clear. Thank you

companies:
  type: pandas.CSVDataset
  filepath: ${_base_location}/01_raw/companies.csv
  metadata:
    kedro-viz:
      layer: raw
      preview_args:
        nrows: 5

@stephkaiser
Copy link

@ravi-kumar-pilla understood, thank you!

Copy link
Contributor

@rashidakanchwala rashidakanchwala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. thanks! <3

@ravi-kumar-pilla ravi-kumar-pilla merged commit c90e723 into main Sep 10, 2024
25 checks passed
@ravi-kumar-pilla ravi-kumar-pilla deleted the chore/lite-metatdata-change branch September 10, 2024 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants