Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flatten effects #1837

Merged
merged 10 commits into from
Aug 15, 2024
Merged

Conversation

slinkydeveloper
Copy link
Contributor

@slinkydeveloper slinkydeveloper commented Aug 14, 2024

Depends on #1828, replaces #1620

This PR flattens the effect logic straight into the business logic of the state machine. Check the individual commits for more info.

No business logic was changed in any way, and there's room to do more cleanup. There's also no logical reason for having separate files command_handler.rs and effect_interpreter.rs, I simply tried to reduce the number of changes. I think going forward we should instead reorganize the business logic on a "command basis", that is all code related to the invoke command goes in handle_command.rs and stuff like this (and related tests too). But we can do this iteratively.

As followup we can also do #276, which is probably trivial at this point.

Copy link
Contributor

@tillrohrmann tillrohrmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for unifying the EffectInterpreter and the CommandInterpreter @slinkydeveloper. The changes look good to me. I only had a few minor comments/questions. I think it would be great to unify the different StateMachine impl blocks that are spread across a few files now as a follow-up. Apart from this, +1 for merging.

pub(crate) struct EffectInterpreter<Codec> {
_codec: PhantomData<Codec>,
}
impl<Codec: RawEntryCodec> StateMachine<Codec> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that this is a temporary state and thus this comment is for when we move the code: Let's not split the impl block across multiple files. In some parts of our code we did it like this and I always find it very surprising finding some more code in some seemingly unrelated files.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm gonna merge this all together in this PR now and merge.

crates/worker/src/partition/state_machine/mod.rs Outdated Show resolved Hide resolved
crates/worker/src/partition/state_machine/mod.rs Outdated Show resolved Hide resolved
Ok(())
}

#[test(tokio::test(flavor = "multi_thread", worker_threads = 2))]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific reason why the number of worker threads is 2 and we are not using the current thread runtime?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IDK about this, this change was brought in by @AhmedSoliman, I'm simply following the way the other tests were changed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not needed anymore. There was a time during a transition when that was required. Feel free to change it back if needed.

}

#[test(tokio::test)]
async fn send_response_using_invocation_id() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which test covers this test case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Basically every test where we send completions, which is a good part of the other tests...

…the Command interpreter and squashes together apply and effect interpret. on_apply is implemented in the next commits.
… flattening of the debug/trace operation for each effect + the actual interpretation (which was previously in interpret_effect). Some of those methods can be further simplified improved, but for the sake of simplicity to review the PR I tried to keep this refactoring as mechanical as possible, without any business logic changes.

Over the next PRs I'll start moving some of the business logic within the rest of the state machine.
…e do_ methods directly. As the previous commit, some of the business logic can be further simplified/improved, but for the sake of simplicity to review the PR I tried to keep this refactoring as mechanical as possible, without any business logic changes.

The only bit that was changes is the code are the loops in kill_child_invocations and cancel_journal_leaves, as i can't pin anymore the stream b/c of the mutable refs to ctx.storage
The only test that was skipped is send_response_using_invocation_id, as it was redundant.
@slinkydeveloper slinkydeveloper merged commit ef39f7b into restatedev:main Aug 15, 2024
7 checks passed
@slinkydeveloper slinkydeveloper deleted the flatten-effects branch August 15, 2024 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants