Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Demo slicing pipeline final #2036

Merged
merged 182 commits into from
Sep 6, 2024
Merged

Demo slicing pipeline final #2036

merged 182 commits into from
Sep 6, 2024

Conversation

Huongg
Copy link
Contributor

@Huongg Huongg commented Aug 12, 2024

Description

This PR is for QA-ing purpose.

I've merged all the latest changes from 3 different PRs here (#1980 , #1996, #2003)

QA notes

This PR should cover the following tasks

As a Kedro user, I want to be able to select multiple nodes within the flowchart so that I can slice the pipeline I am currently viewing

  • Notification on First Node Click: When I click on the first node, a notification should appear. This behavior should not occur when clicking on a pipeline (is it referred to as "pipeline" or "modular pipeline"?).
  • Second Node Click with Shift: If I hold the Shift key and click on a second node, the metadata should close immediately, and the notification should disappear.
  • when selecting a node from bottom to top, the order should be updated according to --from-node=bottom-node-id --to-node=top-node-id

As a Kedro user, I want to preview which nodes are included in my selection for slicing so that I can understand what my slice will be before creating a new slice

  • Chart Highlighting: The two nodes selected with Shift held should be highlighted with a darker blue background, and any nodes in between should be highlighted with a lighter blue background.
  • Nodes List Highlighting: The nodes list on the left-hand side should also highlight all the nodes included in the newly selected pipeline.
  • Call to Action: A button at the bottom should display the number of nodes selected and include a "Slice" button.
  • Outside Click: Clicking outside the selection will un-select all the nodes, and go back to the default/previous page. This includes clicking on any other nodes outside the sliced pipeline

As a Kedro user, I want to view my sliced pipeline and its corresponding run command so that I can run this slice in the CLI for my Kedro project. I also want to be able to reset my sliced pipeline view so that I can go back to viewing the full Kedro pipeline

  • Filtered Nodes View: A new view should display all the filtered nodes.
  • Updated Node List: The node list should reflect all the newly filtered nodes.
  • CLI Command Display: A command line at the bottom should show how to run the filtered nodes in the CLI.
  • Close the sliced view: Click the exit button should take the user back to viewing the full kedro pipeline

Checklist

  • Read the contributing guidelines
  • Opened this PR as a 'Draft Pull Request' if it is work-in-progress
  • Updated the documentation to reflect the code changes
  • Added new entries to the RELEASE.md file
  • Added tests to cover my changes

Huong Nguyen added 30 commits June 14, 2024 22:08
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Huong Nguyen added 3 commits August 23, 2024 10:22
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
@Huongg
Copy link
Contributor Author

Huongg commented Aug 23, 2024

hey @astrojuanlu thanks for reviewing the PR. You've spotted a very good bug there and as we discussed, I've addressed the command to only show --to-output or --to-nodes. Additionally, the missing command line is now included even when the node lacks a user-specified name.

  • kedro run --to-outputs
Screenshot 2024-08-23 at 17 21 06

-kedro run --to-node
Screenshot 2024-08-23 at 17 20 57

@jitu5
Copy link
Contributor

jitu5 commented Aug 29, 2024

@Huongg @noklam @rashidakanchwala
I tested the slicing feature in the VSCode extension. In terms of functionality, it is working fine, but there are some issues related to the UI.

On first click on node, the Notification is not coming in center.
Screenshot 2024-08-29 at 5 36 40 p m

After slicing, the action bar at the bottom including run command and slicing button is hidden.
Screenshot 2024-08-29 at 5 37 14 p m

If the same window is opened in full column layout, the entire action bar will be visible.
Screenshot 2024-08-29 at 5 38 18 p m

Same for the sliced window.
Screenshot 2024-08-29 at 5 42 12 p m

When the Kedro Viz window is opened for the first time, it appears in the second column layout and not in full screen. Considering this, we need to either fix the 'Notification on First Node Click' or disable the slicing feature for at least the 0.0.3 Kedro-VSCode release.

Gif:
slicing-vscode

@Huongg
Copy link
Contributor Author

Huongg commented Sep 3, 2024

thanks @jitu5 I can see why this happened. At the moment the way I calculate the position of the box is based on the width constant of sidebar, but i guess it's not the same value for the vscode. Good spot, and and thanks for checking this. Will have a think about how we should address this

@stephkaiser
Copy link

Thanks @Huongg for addressing my comments! slicing is looking great and we are getting closer

Some minor comments from me:

  • btw the animation looks great!

  • sometimes child nodes are still not highlighted at parent node group level (it only shows highlight at parent level if all child nodes in the group are selected)

Screenshot 2024-09-03 at 12 40 34
  • I wasn't able to test/see what the gradients look like when a run command is really long since it is much shorter now, maybe we can have a quick call about this

  • after slicing, when clicking on a node in sliced view and metadata panel opens, the position of the run command bar is not in the center, but it is perfect when closing the left list panel:

Screenshot 2024-09-03 at 12 43 40
  • Question on changing the run command to only --to-outputs, is it very difficult to solve the run command issue? my concern is that the value and usefulness of this feature for users would now be less since the purpose of slicing is so they can run a specific part of the pipeline but if users don't have full control over that then users might not find enough value in using this feature. Wanted to hear your thoughts too @astrojuanlu

@Huongg
Copy link
Contributor Author

Huongg commented Sep 3, 2024

@stephkaiser yeah let's have a chat, I can show you the shadow box and also the context for the run command challenge we have.

jitu5 and others added 6 commits September 3, 2024 16:11
* Disable slicing feature in embedded mode

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* test fix

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Name fix

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Moved visibleSlicing prop under visible

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Test fix

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

* Remove unused code

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>

---------

Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com>
* adding data-test

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* include getDataTestAttribute

Signed-off-by: Huong Nguyen <huongg1409@gmail>

---------

Signed-off-by: Huong Nguyen <huongg1409@gmail>
Co-authored-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
@Huongg
Copy link
Contributor Author

Huongg commented Sep 4, 2024

hey @stephkaiser as discussed, I've addressed 2 points about the position of the run command, and the modular pipeline not highlighted when collapsed. Will create the ticket for the box shadow separately. Let me know if you're happy the changes or found some other issue 😄

@stephkaiser
Copy link

it looks good to me! I have no more comments, thank you for all the hard work @Huongg!

Copy link
Contributor

@rashidakanchwala rashidakanchwala left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazzzzing ! @Huongg

Huongg and others added 3 commits September 5, 2024 12:43
* first draft

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* fix typo

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* include gif

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* use uncapital words

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* remove the word just

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* remove #

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* use verb instead of slicing

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* use stable version

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* include new gif and increase the size

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* remove un-used gif

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* remove ./ in path

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* revert back to previous syntax

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* reuploaded gif files

Signed-off-by: Huong Nguyen <huongg1409@gmail>

* update text

Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>
Signed-off-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com>

* update text

Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>
Signed-off-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com>

* update text

Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>
Signed-off-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com>

* update text

Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>
Signed-off-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com>

---------

Signed-off-by: Huong Nguyen <huongg1409@gmail>
Signed-off-by: Huong Nguyen <32060364+Huongg@users.noreply.github.com>
Co-authored-by: Huong Nguyen <huongg1409@gmail>
Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>
Signed-off-by: Huong Nguyen <huongg1409@gmail>
Copy link
Contributor

@SajidAlamQB SajidAlamQB left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work @Huongg! 🔥

Copy link
Contributor

@jitu5 jitu5 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Huongg. amazing work! 🔥🔥

Copy link
Contributor

@ravi-kumar-pilla ravi-kumar-pilla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work @Huongg and @stephkaiser !!

Copy link
Member

@astrojuanlu astrojuanlu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was finally able to test this ❤️ Well done @Huongg!

@Huongg Huongg merged commit cc1a119 into main Sep 6, 2024
25 checks passed
@Huongg Huongg deleted the demo-slicing-pipeline-final branch September 6, 2024 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants