Custom mouse behavior

[lchauvin]

Hello,

I would like to create some custom behavior for mouse events.
I want for example to have right click to activate / deactivate the crosshair (right click to activate, stay press, move, then release to deactivate), left click for painting, and middle button for pan / zoom.

What would be the best way to do that? Use the current tools and tweak the mouse behaviors, or create a new tool, that will encompass all of these features, and set its own behavior?

Thank you.

[lchauvin]

What do you think would be the best approach ?

[PaulHax]

That sounds like a good approach. Some code assosated with VtkTwoView needs to cache the mouse position (as keyboard event does not have it.) Then on c, trigger some update on the crosshairs widget.

Could be nice to get this fucntion to fire which converts from mouse coordinates to "world space":
https://github.com/Kitware/VolView/blob/main/src/vtk/CrosshairsWidget/behavior.ts#L48
Perhaps by asking the VTK.js WidgetManager class to process a synthesized mouse event from the cached mouse position.

(Also, looks like there is a bug if I press c twice...)

[lchauvin]

Yes, I've been trying to trigger this function with a mouse event, but couldn't do it. I couldn't find a way to call it directly from VtkTwoView either. I'm not sure how to access the behavior from the WidgetManager (sorry if it's trivial, I'm not so familiar with the code yet).

[PaulHax]

I'm learning too =)

This function should a trigger the crosshairs behaivor widget:
https://github.com/Kitware/vtk-js/blob/master/Sources/Widgets/Core/WidgetManager/index.js#L186

Looks like the vtk interactor calls it so maybe we need to poke an interactor somehow:
https://github.com/Kitware/vtk-js/blob/master/Sources/Widgets/Core/WidgetManager/index.js#L286-L289

Looks like View structures have an handle to interactors:
https://github.com/Kitware/VolView/blob/main/src/components/tools/paint/PaintWidget2D.vue#L134

I'm not familar with vtk.js interactors. =(

[floryst]

Just like Paul mentioned, there needs to be a "caching" behavior for the mouse position that then gets passed to the initialization of the tool. This should most likely be done in the vtkCrosshairsWidget behavior code.

[lchauvin]

I also noticed this comment in vtkCrosshairWidget behavior:

  publicAPI.handleMouseMove = (callData: any) => {
    // technically need to call requestAnimation during
    // the initial phase of placing the crosshairs,
    // but that's not needed since no VTK object
    // is actually being rendered.

I'm not sure if it's related to the same issue or not.

[floryst]

Having custom mouse behavior is a generally good thing to have! While I don't have an exact pathway for this, I have two thoughts on it that I'll revisit later.

• The vtk.js widgets should support configurable actions directly. A concrete example of this would be binding crosshair interactions to left mouse or right mouse buttons.
• A vtkInteractorObserver middleware that supports configurable button translations. A concrete example of this would be to translate right buttons to left button events, passing on the left button event to downstream observers.