Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: peer automatically injects existing task metadata to scheduler upon restart #3385

Closed
wants to merge 1 commit into from

Conversation

BruceAko
Copy link
Contributor

@BruceAko BruceAko commented Jul 19, 2024

Description

Handle AnnouncePeersRequest, store Peer into scheduler.

Related Issue

Resolve #3387

Motivation and Context

Rust's dfdaemon stores the task's metadata in RocksDB at the beginning of the download. Add a new rpc interface: AnnounceTask() to the v2 scheduler. dfdaemon of Rust will fetch the metadata of downloaded tasks (including the metadata of tasks and pieces) from RocksDB at each startup, filter out all the completed tasks, call AnnounceTask() to the schedulerscheduler to add the metadata of Task and Peer to the Manager for subsequent scheduling.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation Update (if none of the other choices apply)

Checklist

  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.

Test doc

https://www.yuque.com/baimo/fv9qk8/vw6mgd2aw0ksv0c8

@BruceAko BruceAko requested a review from a team as a code owner July 19, 2024 08:10
Copy link

codecov bot commented Jul 19, 2024

Codecov Report

Attention: Patch coverage is 67.39130% with 30 lines in your changes missing coverage. Please review.

Project coverage is 52.07%. Comparing base (2c0ae78) to head (d353508).
Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
scheduler/service/service_v2.go 70.45% 25 Missing and 1 partial ⚠️
scheduler/rpcserver/scheduler_server_v2.go 0.00% 4 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3385      +/-   ##
==========================================
- Coverage   52.11%   52.07%   -0.04%     
==========================================
  Files         188      188              
  Lines       20242    20332      +90     
==========================================
+ Hits        10549    10588      +39     
- Misses       8894     8940      +46     
- Partials      799      804       +5     
Flag Coverage Δ
unittests 52.07% <67.39%> (-0.04%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scheduler/metrics/metrics.go 8.24% <ø> (ø)
scheduler/rpcserver/scheduler_server_v2.go 2.56% <0.00%> (-0.14%) ⬇️
scheduler/service/service_v2.go 81.40% <70.45%> (-1.00%) ⬇️

... and 5 files with indirect coverage changes

@BruceAko BruceAko force-pushed the main branch 4 times, most recently from aea7f27 to 14cedf3 Compare July 23, 2024 04:50
Copy link
Member

@gaius-qi gaius-qi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PeerCountLimitForTask

@BruceAko BruceAko force-pushed the main branch 5 times, most recently from 22715c2 to 52669a1 Compare August 2, 2024 03:27
@BruceAko BruceAko changed the title [WIP]feat: peer automatically injects existing task metadata to scheduler upon restart feat: peer automatically injects existing task metadata to scheduler upon restart Aug 2, 2024
@BruceAko BruceAko requested a review from gaius-qi August 2, 2024 03:32
@BruceAko
Copy link
Contributor Author

BruceAko commented Aug 2, 2024

PeerCountLimitForTask

resolved

@BruceAko BruceAko changed the title feat: peer automatically injects existing task metadata to scheduler upon restart [WIP] feat: peer automatically injects existing task metadata to scheduler upon restart Aug 26, 2024
@BruceAko BruceAko changed the title [WIP] feat: peer automatically injects existing task metadata to scheduler upon restart feat: peer automatically injects existing task metadata to scheduler upon restart Aug 29, 2024
@BruceAko BruceAko changed the title feat: peer automatically injects existing task metadata to scheduler upon restart [WIP] feat: peer automatically injects existing task metadata to scheduler upon restart Aug 29, 2024
Signed-off-by: BruceAko <chongzhi@hust.edu.cn>
@BruceAko BruceAko changed the title [WIP] feat: peer automatically injects existing task metadata to scheduler upon restart feat: peer automatically injects existing task metadata to scheduler upon restart Aug 29, 2024
@gaius-qi gaius-qi closed this Sep 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Peer automatically injects existing task metadata to scheduler upon restart.
3 participants