-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: a new ChainIndexer to index tipsets, messages and events #12421
Open
aarshkshah1992
wants to merge
118
commits into
master
Choose a base branch
from
feat/msg-eth-tx-index
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+6,270
−5,407
Open
Changes from 112 commits
Commits
Show all changes
118 commits
Select commit
Hold shift + click to select a range
fb19d79
chain index complete for msgs and txns
aarshkshah1992 58569d6
dont need observer changes for now
aarshkshah1992 5a3f76f
changes
aarshkshah1992 9ea48f3
fix tests
aarshkshah1992 1e3a9d5
fix tests
aarshkshah1992 4c34bc7
use th right context
aarshkshah1992 285ce26
index empty tipsets correctly
aarshkshah1992 12e67fe
implement automated backfilling
aarshkshah1992 3377987
add event indexing and remove all old indices
aarshkshah1992 e6331da
fix test
aarshkshah1992 f1f24c8
revert deployment test changes
aarshkshah1992 3f09e1e
revert test changes and better error handling for eth tx index lookups
aarshkshah1992 c3865b7
fix sql statments naming convention
aarshkshah1992 8ec7cd2
address review for Index GC
aarshkshah1992 a1c5201
more changes as per review
aarshkshah1992 e0831ee
changes as per review
aarshkshah1992 06ca87a
fix config
aarshkshah1992 c30079e
mark events as reverted during reconciliation
aarshkshah1992 01d78e3
better reconciliation; pens down and code complete; also reconcile ev…
aarshkshah1992 994a717
fix tests
aarshkshah1992 ad9bcb1
improve config and docs
aarshkshah1992 0c9a0ca
improve docs and error handling
aarshkshah1992 89cedb2
improve read logic
aarshkshah1992 a2a2b76
improve docs
aarshkshah1992 44af9f8
better logging and handle ennable event storage
aarshkshah1992 6608b80
improve logs and index init proc
aarshkshah1992 625d8c8
better logging
aarshkshah1992 fed08b0
fix bugs based on calibnet testing
aarshkshah1992 a5c56c1
create sqliite Indices
aarshkshah1992 7acd481
gc should be based on epochs
aarshkshah1992 821dcd4
fix event query
aarshkshah1992 cde46cb
foreign keys should be enabled on the DB
aarshkshah1992 727dae3
reverted tipsets should be removed as part of GC
aarshkshah1992 c07784d
release read lock
aarshkshah1992 896048a
make it easy to backfill an empty index using reconciliation
aarshkshah1992 602f660
better docs for reconciliation
aarshkshah1992 9750571
Merge remote-tracking branch 'origin/master' into feat/msg-eth-tx-index
aarshkshah1992 13c2824
fix conflicts with master
aarshkshah1992 37d6746
Apply suggestions from code review
aarshkshah1992 c4490bb
fix go mod
aarshkshah1992 93a8b76
fix formatting
aarshkshah1992 6f8530e
revert config changes
aarshkshah1992 627aff2
address changes in observer
aarshkshah1992 7244b66
remove top level chainindex package
aarshkshah1992 531cd38
changes as per review
aarshkshah1992 77fc462
changes as per review
aarshkshah1992 c2e5f68
changes as per review
aarshkshah1992 286af22
handle index with reverted tipsets during reconciliation
aarshkshah1992 d67a30a
changes as per review
aarshkshah1992 5f5ef3a
fix type of max reconcile epoch
aarshkshah1992 f5a5c61
changes to reconciliation as per review
aarshkshah1992 730d00a
log ipld error
aarshkshah1992 c099abf
better logging of progress
aarshkshah1992 951ce77
disable chain indexer hydrate from snapshot based on config
aarshkshah1992 ad6c086
always populate index
aarshkshah1992 52e104d
make config easy to reason about
aarshkshah1992 9c6c728
fix config
aarshkshah1992 1da1e07
fix messaging
aarshkshah1992 efe90f8
revert config changes
aarshkshah1992 c121321
Apply suggestions from code review
aarshkshah1992 2ff1d42
changes as per review
aarshkshah1992 c945bb5
make error messages homogenous
aarshkshah1992 432e09a
fix indentation
aarshkshah1992 af9bc23
changes as per review
aarshkshah1992 6d84b03
feat: recompute tipset to generate missing events if event indexing i…
aarshkshah1992 b9f1583
better docs for gc retention epoch
aarshkshah1992 1921abd
imrpove DB handling (#12485)
aarshkshah1992 c03c4c7
Merge branch 'master' into feat/msg-eth-tx-index
aarshkshah1992 b7966e3
fix conflict
aarshkshah1992 da76042
fix lite node config for indexer
aarshkshah1992 1e8f8ce
exclude reverted events from eth get logs if client queries by epoch
aarshkshah1992 7b304a4
Simply addressing for event lookups in the index.
aarshkshah1992 5e2dba4
Merge remote-tracking branch 'origin/master' into feat/msg-eth-tx-index
aarshkshah1992 593e724
Apply suggestions from code review
aarshkshah1992 aa2aa4c
Apply suggestions from code review
aarshkshah1992 0339c46
fix tests
aarshkshah1992 105d85b
Apply suggestions from code review
aarshkshah1992 a9788f5
feat: migration("re-indexing"), backfilling and diasgnostics tooling …
aarshkshah1992 fcb6012
fix lint
aarshkshah1992 e3be131
Apply suggestions from code review
aarshkshah1992 8f0ee3c
Apply suggestions from code review
aarshkshah1992 04ac694
remove reverted flag from RPC
aarshkshah1992 f92b03c
Apply suggestions from code review
aarshkshah1992 5b58fe5
fix testing of events and dummy chain store
aarshkshah1992 86a8d26
remove lotus shed commands for old Indices
aarshkshah1992 aff1a6c
change type of event counts to uint64
aarshkshah1992 fe76994
only recompute events if theyre not found
aarshkshah1992 361d7bf
short-circuit empty events path for older tipsets
aarshkshah1992 f628cb2
chain indexer must be enabled if ETH RPC is enabled
aarshkshah1992 c96f4ed
change name of message_id column to id in tipset_message table
aarshkshah1992 cb7150c
only expose SetRecomputeTipSetStateFunc
aarshkshah1992 5575a1d
dont block on head indexing for reading messages
aarshkshah1992 e2013a0
document why we're only checking for missing events for a single tipset
aarshkshah1992 fcda11b
document when we query for reverted events
aarshkshah1992 ee6e589
simplify event collection
aarshkshah1992 8aad8e4
Apply suggestions from code review
aarshkshah1992 ab4de90
fix test
aarshkshah1992 bc97ebe
change event_id to id in the event table
aarshkshah1992 06e139f
change head indexed timeout
aarshkshah1992 947e6d1
remove deprecated config options
aarshkshah1992 5887cdf
fail ETH RPC calls if ChainIndexer is disabled
aarshkshah1992 03627aa
fix docs
aarshkshah1992 b4939ae
remove the tipset key cid func from lotus shed
aarshkshah1992 091574c
Merge remote-tracking branch 'origin/master' into feat/msg-eth-tx-index
aarshkshah1992 54d6632
address review comments
aarshkshah1992 1135da7
Apply suggestions from code review
aarshkshah1992 6dd3fcc
chore(events): remove unnecessary DisableRealTimeFilterAPI (#12610)
rvagg 16123bc
feat(cli): add --quiet to chainindex validate-backfill + cleanups (#1…
rvagg 488202b
fix tests
aarshkshah1992 8e6229e
Apply suggestions from code review
aarshkshah1992 5836ccb
error type for disabled chainindexer
aarshkshah1992 d0fb6e2
Merge remote-tracking branch 'origin/master' into feat/msg-eth-tx-index
aarshkshah1992 70aaef4
fix(chainindex): recompute tipset when we find no receipts
rvagg 702f1f5
fix(chainindexer): backfilling should halt when chain state data is m…
aarshkshah1992 bd72eec
reduce log noise
aarshkshah1992 c3937d7
Merge remote-tracking branch 'origin/master' into feat/msg-eth-tx-index
aarshkshah1992 0311749
make jen
aarshkshah1992 fe16f30
make jen
aarshkshah1992 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
blocking merge with this comment because this needs to be updated by the docs PR