[Feature suggestion] Integrate something like demucs as a preprocessing stage before Whisper

[justinkb]

I find it helps to isolate the actual spoken voices from videos first, before letting whisper do its thing

[niksedk]

Purfview Faster Whisper XXL has several VAD (Voice Activity Detection) engines included.

[justinkb]

thanks for pointing me in that direction, unfortunately I don't think their builds support AMD gpus through HIP. I'm currently using whisper.cpp built for HIP SDK, while preprocessing using wsl rocm builds of pytorch

as a workaround, maybe you could add an option to define our own whisper invocations with placeholder variables. that way I can invoke it from Windows into a wsl distro like for example;

wsl -d Ubuntu-22.04 -- source /home/paul/dev/venvs/ml/bin/activate `&`& python3 my-fw-script.py %videofile% %output

if you don't want to do that, I understand, I will implement it locally and build my own subtitleedit.