Toy chat fix #105

Ivansete-status · 2024-12-19T22:07:47Z

This PR aims for recovering toy chat powered by libwaku/nwaku.

Aside from that, it suggests to make the waku crate to behave tokio-asynchronously.

Note: I've noticed that the libwaku cannot be used by multiple threads in parallel. We might need to perform additional work to make it work properly:

In the following snippet, the initialized variable is shared among all threads. We should instead have a mechanism to check "initialized" within each thread.

proc initializeLibrary() {.exported.} =
  if not initialized.exchange(true): <-----------
    NimMain() # Every Nim library needs to call `NimMain` once exactly
  when declared(setupForeignThreadGc):
    setupForeignThreadGc()
  when declared(nimGC_setStackBottom):
    var locals {.volatile, noinit.}: pointer
    locals = addr(locals)
    nimGC_setStackBottom(locals)

danielSanchezQ

I think its nice, I would check on my comment to see if it can be improved. But nice job overall.

danielSanchezQ · 2024-12-20T09:46:18Z

waku-bindings/src/node/filter.rs

+    let mut result = LibwakuResponse::default();
+    let notify = Arc::new(Notify::new());
+    let notify_clone = notify.clone();
+    let result_cb = |r: LibwakuResponse| {
+        result = r;
+        notify_clone.notify_one(); // Notify that the value has been updated
+    };


I think this pattern its ok but should be simpler if you use a channel to return the result. You can probably use kanal a mixed flavored channel (sync/async). To simplify it.

danielSanchezQ · 2024-12-20T09:51:12Z

@Ivansete-status Are the nwaku calls on the other side of the FFI are async?

Ivansete-status · 2024-12-20T10:26:04Z

@Ivansete-status Are the nwaku calls on the other side of the FFI are async?

Yes. Everything in nwaku is async and the interaction with the library is async too. To elaborate a bit more, libwaku runs a separate thread, a.k.a. The Waku Thread, which is in charge of running the Waku node and also, attending the app's requests (waku_new, waku_start, etc.)

richard-ramos · 2024-12-20T13:31:02Z

waku-sys/build.rs

@@ -68,12 +68,6 @@ fn generate_bindgen_code(project_dir: &Path) {

    println!("cargo:rustc-link-lib=stdc++");

-    println!(


Negentropy is no longer needed to build libwaku?! that's very cool!

richard-ramos · 2024-12-20T18:12:22Z

examples/tic-tac-toe-gui/src/main.rs

+        // let ctopic = WakuContentTopic::new("waku", "2", "tictactoegame", Encoding::Proto);
+        // let content_topics = vec![ctopic];
+        // waku.filter_subscribe(&self.game_topic, content_topics).await.expect("waku should subscribe");


Maybe a comment can be added above these lines to explain what is their purpose (i.e. to indicate that filter subscriptions are done with it)

richard-ramos · 2024-12-20T18:12:44Z

examples/tic-tac-toe-gui/src/main.rs

+            dbg!(format!("message hash published: {}", msg_hash));
+        }
+
+        // self.waku.lightpush_publish_message(&message, &self.game_topic);


Same comment as before. Add a comment to indicate what is this about

richard-ramos · 2024-12-20T18:15:57Z

waku-bindings/src/node/events.rs

-        let s = "{\"eventType\":\"message\",\"messageHash\":\"0x26ff3d7fbc950ea2158ce62fd76fd745eee0323c9eac23d0713843b0f04ea27c\",\"pubsubTopic\":\"/waku/2/default-waku/proto\",\"wakuMessage\":{\"payload\":\"SGkgZnJvbSDwn6aAIQ==\",\"contentTopic\":\"/toychat/2/huilong/proto\",\"timestamp\":1665580926660}}";
-        let evt: Event = serde_json::from_str(s).unwrap();
-        assert!(matches!(evt, Event::WakuMessage(_)));
+        let s = "{\"eventType\":\"message\",\"messageHash\":[91, 70, 26, 8, 141, 232, 150, 200, 26, 206, 224, 175, 249, 74, 61, 140, 231, 126, 224, 160, 91, 80, 162, 65, 250, 171, 84, 149, 133, 110, 214, 101],\"pubsubTopic\":\"/waku/2/default-waku/proto\",\"wakuMessage\":{\"payload\":\"SGkgZnJvbSDwn6aAIQ==\",\"contentTopic\":\"/toychat/2/huilong/proto\",\"timestamp\":1665580926660}}";


Why use a numeric array instead of a byte array?
reason why I ask is that one of the advantages of json is that it's human readable (not much tbf), Then, comparing the byte array with the hash, it seems to me that a hash is much more friendly.
(Although tbf, we do use a MessageHash type that abstracts the dev from the array)

@richard-ramos - yes good point and I completely agree the need of having readable hex.
I found this is the most convenient approach of handling message hash from store and relay. Also, in the messagehash.rs module we have the following, which is interesting to "print" the msg_hash content:

// Implement the Display trait impl fmt::Display for MessageHash { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{}", self.to_hex_string()) } }

richard-ramos · 2024-12-20T18:17:36Z

waku-bindings/src/node/filter.rs

+    let mut result = LibwakuResponse::default();
+    let notify = Arc::new(Notify::new());
+    let notify_clone = notify.clone();
+    let result_cb = |r: LibwakuResponse| {
+        result = r;
+        notify_clone.notify_one(); // Notify that the value has been updated
+    };


This patterns tends to repeat consistently. I wonder if we can maybe extract this into a function?

richard-ramos · 2024-12-20T18:20:20Z

waku-bindings/src/node/mod.rs

+        let one_day_in_secs = 60 * 60 * 24;
+        let time_start = (Duration::from_secs(Utc::now().timestamp() as u64)
+            - Duration::from_secs(one_day_in_secs))
+        .as_nanos() as usize;


Shoudn't the time-start and time-end be optional duration's passed by the developer?

richard-ramos · 2024-12-20T18:21:47Z

waku-bindings/src/node/mod.rs

+                true,             // pagination_forward
+                Some(25),         // pagination_limit,


Should probably be parameters as well.

Perhaps for all of the parameters of the query it makes sense to define a query object that can be built with an options pattern (https://medium.com/@omid.jn/options-pattern-in-rust-6425520b2b23)

richard-ramos · 2024-12-20T18:22:18Z

waku-bindings/src/node/mod.rs

+                true,             // pagination_forward
+                Some(25),         // pagination_limit,
+                peer_addr,
+                None, // timeout_millis


Timeout should probably be a parameter given by the developer

richard-ramos · 2024-12-20T18:25:45Z

waku-bindings/src/node/store.rs

+#[serde(rename_all = "camelCase")]
+pub struct StoreWakuMessageResponse {
+    pub message_hash: MessageHash,
+    pub message: WakuStoreRespMessage,


IIRC, include_data will determine whether a message will be returned or not, so probably makes sense for message to be an option

Yes correct :) We are using the include_data concept in the requests. I think is fine unless I'm missing something

But wouldn't that mean that if you send include_data: false, your message will contain a (invalid) struct with default values? having it as an option perhaps would make it clearer for the dev that there's no message available

richard-ramos · 2024-12-20T18:26:09Z

waku-bindings/src/general/mod.rs

+    pub timestamp: usize,
+    #[serde(default)]
+    pub ephemeral: bool,
+    // pub proof: Vec<u8>,


Should this be uncommented?

Ivansete-status added 9 commits November 29, 2024 16:19

update waku-sys to latest nwaku master

ad0eab2

some changes to add toy-chat example. not completed yet

0d37037

update waku-sys/vendor to libwaku-store-enhancements branch

be987f8

progress in toy-chat

3f72fc7

proper use of messagehash and store

345dcbe

add nimcache folder to gitignore

51ea3be

compilation fixes and update wakusys/vendor

a5338bf

waku-sys update

9d73660

wide adaptations to make the waku crate behave tokio-based async

e937e05

github-actions bot assigned Ivansete-status Dec 19, 2024

danielSanchezQ reviewed Dec 20, 2024

View reviewed changes

Ivansete-status mentioned this pull request Dec 20, 2024

chore: enhance libwaku store protocol and more waku-org/nwaku#3223

Open

Ivansete-status added 2 commits December 20, 2024 18:05

make sure the tests run in serial and the waku_destroy is called

666bc73

simple changes to make clippy happier

dd29900

richard-ramos reviewed Dec 20, 2024

View reviewed changes

Ivansete-status added 12 commits December 22, 2024 14:42

tic-tac-toe main add some more comments

f27cda7

avoid duplicated code and better separate structs in modules

08b8190

uncomment proof attribute from WakuStoreRespMessage

14aa84f

use uuid instead of hard-coded request id in store query

eb4286b

expose include_data to expert devs for store requests fn

40e908f

better time management

3cbb259

apply better option approach in store queries

80ea725

fix test compilation issue and add some more time for waku

f2a6c8e

tic-tac-toe minor fixes

5b0204f

avoid warnings

863f208

avoid clippy errors

1052dca

simple change to avoid fmt warning

a6a5988

richard-ramos approved these changes Dec 23, 2024

View reviewed changes

Ivansete-status added 5 commits December 23, 2024 15:26

waku-sys build rm dependency with libc++ avoiding macos issues

2544150

clippy changes

cc76ac0

add on_exit todo so that the waku node is stopped correctly

35bacf4

build.rs cleanup avoid linking ssl crypto and pthread

e25050e

general mod.rs remove empty line to avoid clippy picky issue

2d507c4

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Toy chat fix #105

Toy chat fix #105

Ivansete-status commented Dec 19, 2024 •

edited

Loading

danielSanchezQ left a comment

danielSanchezQ Dec 20, 2024

danielSanchezQ commented Dec 20, 2024

Ivansete-status commented Dec 20, 2024

richard-ramos Dec 20, 2024

richard-ramos Dec 20, 2024

richard-ramos Dec 20, 2024

richard-ramos Dec 20, 2024

Ivansete-status Dec 22, 2024

richard-ramos Dec 20, 2024

richard-ramos Dec 20, 2024

richard-ramos Dec 20, 2024

richard-ramos Dec 20, 2024

richard-ramos Dec 20, 2024

Ivansete-status Dec 22, 2024

richard-ramos Dec 23, 2024

richard-ramos Dec 20, 2024

		@@ -68,12 +68,6 @@ fn generate_bindgen_code(project_dir: &Path) {

		println!("cargo:rustc-link-lib=stdc++");

		println!(

Toy chat fix #105

Are you sure you want to change the base?

Toy chat fix #105

Conversation

Ivansete-status commented Dec 19, 2024 • edited Loading

danielSanchezQ left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

danielSanchezQ commented Dec 20, 2024

Ivansete-status commented Dec 20, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Ivansete-status commented Dec 19, 2024 •

edited

Loading