Upscalers support

[Loukious]

Currently in new versions of MPV, they added support for upscalers. I've compiled a new version of Stremio-Shell to enable mpv.conf support and few other changes to enable d3d11 in Qt. But there seems to be something that sets vo back to opengl-cb (libmpv) but I can't find it to prevent it from doing so.
From mpv.log:

[   1.862][v][cplayer] Set property: no-sub-ass=null -> -3
[   1.862][w][cplayer] Driver 'opengl-cb' has been replaced with 'libmpv'!
[   1.862][v][cplayer] Set property: vo="opengl-cb" -> 1
[   1.862][v][cplayer] Set property: osc="no" -> 1
[   1.862][v][cplayer] Set property: input-defalt-bindings="no" -> -3
[   1.862][v][cplayer] Set property: input-vo-keyboard="no" -> 1
[   1.863][v][cplayer] Set property: hwdec="yes" -> 1
[   1.863][d][cplayer] Run command: loadfile, flags=64, args=[url="http://127.0.0.1:11471/fec8d3a3a6f2ead02bd8f0ebd0dd92e2f0e7d7ee/43", flags="replace", index="-1", options=""]

I've already changed this

stremio-shell/mpv.cpp

Line 150 in f16a7b1

mpv::qt::set_property(mpv, "vo", "libmpv");

Anyone have any idea?

[Zaarrg]

Hard to tell without ur code.

The Stremio shell uses QQuickFramebufferObject to embed mpv.

By design, QQuickFramebufferObject::Renderer::createFramebufferObject() always creates an OpenGL framebuffer (FBO). That means mpv must render via OpenGL (the “opengl-cb” API) to integrate smoothly with the Qt Quick scene graph.

So it might be because of QQuickFramebufferObject::Renderer::createFramebufferObject().

If u change the vo=libmpv to e.g. vo=gpu --gpu-api=d3d11 u have to make sure qt is also running in D3D11mode and as well adjust stuff like createFramebufferObject in mpv.cpp as everything is specific to opengl. So just changing vo= wont work.

Also the current Stremio Shell uses ANGLE meaning the OpenGl calls are translated to d3d11 so at driver level its effectively d3d while to mpv it appears as opengl thats also probably why mpv chose opengl-cb as this is normally auto selected by mpv.,

Those are the most likely options, one but rather unlikely option might be that the stremio web shell video player sets the vo property by sending it to qt via the mpv signal.

---

Sadly dont really know much about mpv and upscalers as i just recently started looking into those as im working towards adding support to stremio for upscaler like 2x_AnimeJaNai V3.

But if u are interested u can check out my fork stremio-desktop-v5. This is built using latest qt6.8.1 with the latest stremio-web v5 UI and latest mpv 0.39.0.

And just recently added mpv.conf support as seen in this release.

---

Also is there any benefit it switching to d3d11 ? As i also thought when making my fork to switch to d3d11 but to have the widest compatibility across systems OpenGL is most suited. Also D3D11 is Windows-exclusive.

And im not sure if there is really any performance benefit in using d3d11. There might be some using advance shaders/upscalers but that isn't the normal use case of the stremio mpv player.

Also for Qt6 for example the embeded mpv player would be needed to be remade using Qt RHI (Also the case for Qt5 just not Qt RHI) which would prevent use of QQuickFramebufferObject and would require manually handling how mpv's D3D textures get composited into the Qt Quick scene graph.

That is also probably why most Qt apps embedding mpv use the simpler OpenGL callback approach as it seamlessly works with QQuickFramebufferObject

But anyway, I hope I could help.

[Loukious]

@Zaarrg thank you for your reply, that's very informative. Yes I tried to render Qt on d3d11 but eventually gave up because I faced an issue with the player being blank (audio only).

Also is there any benefit it switching to d3d11 ?

According to one of MPV devs, to use GPU upscalers, you're required to run Qt in d3d11. So to get my RTX to upscale the video to x2 or even x4, I'll need to get that working.
Also I have to mention that the new Stremio 5.0.0 seems to be written in Rust and the developer said the upcoming version might run on d3d11 (to fix current HDR issues).

[Zaarrg]

@Loukious

Yes I tried to render Qt on d3d11 but eventually gave up because I faced an issue with the player being blank (audio only).

I see, had the exact same issue when i was porting to qt6 and solved it by using:

    QCoreApplication::setAttribute(Qt::AA_ShareOpenGLContexts);
    QQuickWindow::setGraphicsApi(QSGRendererInterface::OpenGL);

This was caused because Qt6 QRhi which auto selected something other then OpenGl in my case it was d3d11. Maybe something similar is happening on ur end.

As well make sure u are using the WID embedding instead of vo=libmpv as the mpv dev suggested as d3d might not be available with vo=libmpv.

Otherwise you would need to check if libmpv works with d3d gpu-api or look for some python, cpp examples of qt with mpv running d3d as i think i saw somewhere qt6 python with d3d mpv.

Also I have to mention that the new Stremio 5.0.0 seems to be written in Rust and the developer said the upcoming version might run on d3d11 (to fix current HDR issues).

Thats interesting, because it would require a lot of changes on Stremio's side to switch from Qt. As for example even in the web ui v5 it uses the Qt Signal to transport events. So it seems like everything related to the Stremio Shell is still built with Qt in mind.

Also in this pr it was mentioned that Qt will probably be used for quite some time, other then on windows as WebView2 could be used there.

Will probably test myself as well to make the embeded mpv player in qt work with d3d. As it would be a nice option to have windows shell run d3d and for macos and linux fallback to opengl.
This would also allow on windows to use the built in mpv upscalers like u said compared to e.g. a webview2 alternative without mpv or qt6 but with opengl mpv.

[Zaarrg]

Will probably test myself as well to make the embeded mpv player in qt work with d3d. As it would be a nice option to have windows shell run d3d and for macos and linux fallback to opengl.

Okay so after some testing libmpv does not support anything other then opengl and there for wont work with d3d11vpp. Even tho setting gpu-api or context as d3d works but it seems like for nvidia upscaling vo has to also be gpu / gpu-next because of that the libmpv implementation based on opengl wont work.

Now embedding mpv using wid works very well. This also works with your settings / the nvidia d3d11vpp upscaling. Sadly with this comes the issue of Stremio using a webview as UI with a QT transparent background.

The issue caused by this is that we cant see through the Webview the emeded mpv player as this wid embedded one is not rendered by QT and there for QT cant show the mpv frames behind the webview. See this stackoverflow issue as mentioned there we would need to use QT to render everything to fix this properly.

So this wont be a trivial matter to implement. I still think that stremio with wid embedded player is the way and will look for a proper solution. As this will fix many issues and allow for full mpv support, any hwdec, any upscaler, proper hdr and everything else that comes with mpv.

[Loukious]

@Zaarrg that's amazing. Looking forward to seeing it fully functional. Thanks a lot.