Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Peer automatically injects existing task metadata to scheduler upon restart. #3387

Open
BruceAko opened this issue Jul 22, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@BruceAko
Copy link
Contributor

BruceAko commented Jul 22, 2024

Description

Rust's dfdaemon stores the task's metadata in RocksDB at the beginning of the download. Add a new rpc interface: AnnounceTask() to the v2 scheduler. dfdaemon of Rust will fetch the metadata of downloaded tasks (including the metadata of tasks and pieces) from RocksDB at each startup, filter out all the completed tasks, call AnnounceTask() to the schedulerscheduler to add the metadata of Task and Peer to the Manager for subsequent scheduling.

Solution

  • Based on the incoming ids and metadata, build Task, Host, and Peer instances in the Manager. If they don't exist, they are created, and if they exist, they are updated.
  • If the Task state is not TaskStateSucceeded, advance the Task state to TaskStateSucceeded and build pieces to be stored in the Task and Peer objects.
  • If the Peer state is not PeerStateSucceeded, advance the Peer state to PeerStateSucceeded.
  • The focus is to first rebuild the Peer's metadata based on the existing Task information and store it in the PeerManager, then add information to the Host and Task that this Peer belongs to, such as adding vertices, InEdge, OutEdge, etc. to the Task.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
1 participant