Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(examples): minimal viable ExEx #7565

Merged
merged 53 commits into from
Apr 11, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
46f6487
wip: exex manager
onbjerg Mar 26, 2024
03f7781
refactor: rename to `spawn`
onbjerg Mar 26, 2024
09e6dda
feat: exex rx channel
onbjerg Mar 28, 2024
9e52e2d
feat: flesh out manager loop
onbjerg Mar 28, 2024
44a1a5f
refactor: move manager to `reth-exex`
onbjerg Apr 8, 2024
d9772d6
refactor: `ExExHandle`
onbjerg Apr 8, 2024
3223632
docs: typo
onbjerg Apr 8, 2024
01494dc
feat: [`ExExHandle::new`]
onbjerg Apr 8, 2024
eeaef8b
fix: retain events > min_id
onbjerg Apr 8, 2024
4af2c9a
feat: most of launch flow
onbjerg Apr 8, 2024
4674ef3
feat: exex id in `install_exex`
onbjerg Apr 8, 2024
3ac8ecd
feat: hook up blockchain tree
onbjerg Apr 8, 2024
4bea51b
chore: rm unused dep
onbjerg Apr 8, 2024
a9a3aab
fix: add id to other `install_exex` impl
onbjerg Apr 8, 2024
6d68a29
fix: use unbounded channels for `ExExEvent`
onbjerg Apr 8, 2024
2086a03
chore: add traces
onbjerg Apr 8, 2024
358d71b
chore: only spawn exex manager if there are exex's
onbjerg Apr 8, 2024
36006b1
docs: move exex docs
onbjerg Apr 8, 2024
3ee6116
feat: basic exex metrics
onbjerg Apr 8, 2024
4ce21fc
feat: handle `ExExEvent`
onbjerg Apr 8, 2024
c4895de
feat: manager metrics
onbjerg Apr 8, 2024
0e8ddc6
refactor: some cleanup
onbjerg Apr 8, 2024
638e474
fix: add missing f64 cast for metrics
onbjerg Apr 8, 2024
242600c
feat: `ExExManagerHandle::ready`
onbjerg Apr 10, 2024
ec91e4e
chore: manager tracing logs
onbjerg Apr 10, 2024
0caefa8
make manager handle mut
shekhirin Apr 10, 2024
18e9691
chore: `AsRef<str>` in `install_exex`
onbjerg Apr 10, 2024
ebd7f16
fix: make it compile
onbjerg Apr 10, 2024
b04012e
fix: actually really make it super compile
onbjerg Apr 10, 2024
25411db
test: make test compile
onbjerg Apr 10, 2024
837a2c9
feat: don't sent notifs exex doesn't care about
onbjerg Apr 10, 2024
4120dfa
feat: `events_sent` metric
onbjerg Apr 10, 2024
c775fa2
docs: finished height docs clarifications
onbjerg Apr 10, 2024
01c360e
fix: various fixes wrt finished height
onbjerg Apr 10, 2024
2d9e71d
chore: rename metrics
onbjerg Apr 10, 2024
fd8a06c
chore: drop span
onbjerg Apr 10, 2024
5c7ea0e
chore: re-introduce span
onbjerg Apr 10, 2024
f237ff9
docs: fix doclinks
onbjerg Apr 10, 2024
879d90f
refactor: rename `block` to `finished_height`
onbjerg Apr 10, 2024
3b5d6f7
feat: `ExExManagerHandle::capacity`
onbjerg Apr 10, 2024
af65a96
chore: stub `poll_ready`
onbjerg Apr 10, 2024
f3ddf17
chore: fmt
onbjerg Apr 10, 2024
e2c36ac
feat: `poll_ready`
onbjerg Apr 10, 2024
a2f670a
chore: clean up
onbjerg Apr 10, 2024
698d6e3
refactor: move exex launch past components builder
onbjerg Apr 10, 2024
e17da42
chore: review
onbjerg Apr 10, 2024
00a26eb
feat(examples): minimal viable ExEx
shekhirin Apr 11, 2024
232cd95
Merge remote-tracking branch 'origin/main' into onbjerg/exex-manager
shekhirin Apr 11, 2024
307bc3f
chore: use `Into<String>`
onbjerg Apr 11, 2024
3648a44
Merge remote-tracking branch 'origin/onbjerg/exex-manager' into alexe…
shekhirin Apr 11, 2024
ab17928
add comment, address PR review
shekhirin Apr 11, 2024
57b8df3
Merge branch 'main' into alexey/minimal-exex
onbjerg Apr 11, 2024
b0645e9
Merge remote-tracking branch 'origin/main' into alexey/minimal-exex
shekhirin Apr 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
chore: clean up
  • Loading branch information
onbjerg committed Apr 10, 2024
commit a2f670ad95c54953d8472b786a8082d24fe19a4d
1 change: 1 addition & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions crates/exex/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ description = "Execution extensions for Reth"
workspace = true

[dependencies]
eyre.workspace = true
futures.workspace = true
metrics.workspace = true
reth-config.workspace = true
Expand Down
20 changes: 6 additions & 14 deletions crates/exex/src/manager.rs
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,8 @@ pub struct ExExHandle {
next_notification_id: usize,

/// The finished block number of the ExEx.
/// tbd describe how this can change, what None means, what this is used for
///
/// If this is `None`, the ExEx has not emitted a `FinishedHeight` event.
finished_height: Option<BlockNumber>,
}

Expand Down Expand Up @@ -128,7 +129,6 @@ pub struct ExExManagerMetrics {
buffer_size: Gauge,
}

// todo: if event is sent to exex it is considered delivered
/// The execution extension manager.
///
/// The manager is responsible for:
Expand All @@ -138,14 +138,9 @@ pub struct ExExManagerMetrics {
/// - Backpressure
/// - Error handling
/// - Monitoring
///
/// TBD
#[derive(Debug)]
pub struct ExExManager {
/// Handles to communicate with the ExEx's.
// todo(onbjerg): we should document that these notifications can include blocks the exex does
// not care about if a longer chain segment is sent - filtering is up to the exex. where do
// we document it, though?
exex_handles: Vec<ExExHandle>,

/// [`CanonStateNotification`] channel from the [`ExExManagerHandle`]s.
Expand Down Expand Up @@ -177,7 +172,7 @@ pub struct ExExManager {
/// The number is inclusive, i.e. all blocks `<= finished_height` are safe to prune.
finished_height: watch::Sender<Option<BlockNumber>>,

/// tbd
/// A handle to the ExEx manager.
handle: ExExManagerHandle,
/// Metrics for the ExEx manager.
metrics: ExExManagerMetrics,
Expand Down Expand Up @@ -257,8 +252,7 @@ impl ExExManager {
}

impl Future for ExExManager {
// todo
type Output = Result<(), ()>;
type Output = eyre::Result<()>;

fn poll(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
// drain handle notifications
Expand All @@ -271,7 +265,6 @@ impl Future for ExExManager {
break
}

// todo: drain blockchain tree notifications
// update capacity
self.update_capacity();

Expand All @@ -288,9 +281,9 @@ impl Future for ExExManager {
.expect("exex expected notification ID outside the manager's range");
if let Some(notification) = self.buffer.get(notification_id) {
debug!(exex.id, notification_id, "sent notification to exex");
if let Poll::Ready(Err(_)) = exex.send(cx, notification) {
if let Poll::Ready(Err(err)) = exex.send(cx, notification) {
// the channel was closed, which is irrecoverable for the manager
return Poll::Ready(Err(()))
return Poll::Ready(Err(err.into()))
}
}
min_id = min_id.min(exex.next_notification_id);
Expand All @@ -317,7 +310,6 @@ impl Future for ExExManager {
}

// update watch channel block number
// todo: clean this up and also is this too expensive
let finished_height = self.exex_handles.iter_mut().try_fold(u64::MAX, |curr, exex| {
let height = match exex.finished_height {
None => return Err(()),
Expand Down