Running scheduled tasks alongside Rocket

[djc-id]

A possibly interesting Reddit thread

https://www.reddit.com/r/rust/comments/qsgxs2/scheduled_background_task_using_database_in_rocket/

I love Rocket but it does need to coexist in a wider ecosystem and there may be an idiom here that is worth documenting.

I don't know enough about Rocket or Tokio to confidently document the optimum solution. Let me know if I can help though.

[tobymurray]

It looks like this is not resolved, on_liftoff needs to return for Rocket to continue so going into an "infinite loop" (a background task that's expected to run for the duration of the application) won't work invoked in the way the post mentions.

[djc-id]

The on_liftoff approach doesn't work but the tokio-cron-scheduler approach mentioned later in that thread does.

[tobymurray]

Using tokio-cron-scheduler is not really Rocket related, which is maybe your original point. It only uses the configuration of Rocket then it's a separate database connection and a separate thread that just so happens to be running near a Rocket instance. Especially if it were to need access to e.g. managed state, a background job like that would get awfully messy pretty quick.

[somehowchris]

@djc-id why would you include basically cron jobs or tasks in an api? It just doesn‘t scale….

It‘s not about rust, rocket or tokio. Let‘s begin with why would you engineer such a thing? Would you eat a blue banana?

• Cronjobs are part of your infrastructure, cf-task-jobs, kWs cronjobs, crontab….
• handling tasks isn‘t the job of an api, if you need to offload jobs, get redis pub/sub, rabbitmq or kafka

[MCOfficer]

A few things that I can't help but reply to.

You mentioned all job scheduling libs/parts for interpreted or IL languages, Rust is not interpreted, not transpiled, has no runtime (besides llvm, which is included)

I feel like I'm missing something here. How is the type of language relevant to how you organize your code?

cargo lets you easily have 2 binaries in a repository, let's say you have the server and a small bin just hitting an endpoint

If that second binary is making a database connection itself, the way OP described, that makes rocket's DB support useless since the user has to manually manage a pool anyways. Or launch a full rocket instance for that second binary.

If it just pings an endpoint on the first instance: yikes! Now that's a hack I didn't want to see today. And it's basically the same as this feature request, it still runs an update on rocket's own threadpool, with rocket's managed state. It's just a more convoluted trigger.

If you feel like ops is a pain, get used to it, do it over and over again, you learn from what you do wrong

If the goal is to become a good overall developer, you're correct. But the goal right here is convenience for the developer. That's why rocket exists in the first place, surely I would learn more by implementing my own server on top of hyper.

[somehowchris]

• Ok so why does the language matter, all these mentioned languages have a runtime, a bottleneck for speed, ressources and agility. Rust doesn‘t have one. A rust binary start (basically) instantly, has no runtime, no heap, nothing to manage per se in general.
• Well if you go for real engineering, the second binary wouldn‘t be allowed to access a db or any ressource directly. It would have to connect to the api server. Now it's important you have a idempotent api for these actions.
• It's not about what your jobs do, its about that you as a dev shouldn't implement when your jobs should run, give that to some OPs component, they are responsible for these kind of things. I don't wanna go further but clocks to trigger jobs inside user-land apps aren't terribly off but they aren't really great.

[MCOfficer]

Ah, I see your point. Still, your solution being a possibility doesn't entirely invalidate the comparison to other Frameworks.

I feel like we've come full circle now. Once again, I believe Rocket should be for hobbyists and prototyping just as much as for professional systems. For the first two cases, your solution is pretty much overengineered, which lines up with my own experience - at least 9/10 of my own hobby projects never make it to a point where scaling is even remotely close.

So the actual question is: How much convenience is too much? Given that other, more business-focused frameworks (f.e. Spring) have also implemented this, I feel like this is fine. It's not like this is some hidden antipattern, either - It's fairly obvious it'll go wrong once you want to scale, and it doesn't complicate switching to a more scalable solution. After all, you're only changing the trigger.

[somehowchris]

Ok so my last few comments may have been a bit harsh on that. As I said in like the my second comment. I see your point. I see that you don't wanna have a super duper scalable system for 100 million+ users. Well then we would talk about totally different things.

But the last few months I have seen code and system which was and were created just from side projects, devs just copied shit over, never though about principal engineering patterns. Since then I have gone another route. If it's not needed by design, don't do it. No matter what it is, a hobby project, work for some company or institution, doesn't matter. If you create the option to do something wrong, i promise you, it will go wrong. (many devs have also no clue how to test properly which makes things go wrong even worse).

TLDR; do as you like, I don't support it per-se, make sure if you document it to mention engineering patterns

[MCOfficer]

That's fair, we can totally agree on "whatever works for you". Offtopic, I feel for you for taking over legacy projects, and I hope you have a nice day :)