Suggestion to add standalone executable for WhisperX

[oep42]

.

[Purfview]

What there is better than just using Faster-Whisper?

[Purfview]

v1 - error, v2 - other error, v3 - some another error... usual Python hell.

Looks like WhisperX is meant for programmers, not for the mortal users. :)
When v2 errors looked like I could fix, I lost interest when I saw that accurate alignment is working only on some languages...

[rsmith02ct]

Thanks for testing it. If they make bug fixes for other languages perhaps worth revisiting.

[Purfview]

Thanks for testing it. If they make bug fixes for other languages perhaps worth revisiting.

Not bug, models are not present at the code: https://github.com/m-bain/whisperX/blob/main/whisperx/alignment.py

[rsmith02ct]

So it's a known limitation and only works with these languages? If it helps with Japanese that would have some benefit as the standard Whisper can be off.

DEFAULT_ALIGN_MODELS_TORCH = {
"en": "WAV2VEC2_ASR_BASE_960H",
"fr": "VOXPOPULI_ASR_BASE_10K_FR",
"de": "VOXPOPULI_ASR_BASE_10K_DE",
"es": "VOXPOPULI_ASR_BASE_10K_ES",
"it": "VOXPOPULI_ASR_BASE_10K_IT",
DEFAULT_ALIGN_MODELS_HF = {
"ja": "jonatasgrosman/wav2vec2-large-xlsr-53-japanese",
"zh": "jonatasgrosman/wav2vec2-large-xlsr-53-chinese-zh-cn",
"nl": "jonatasgrosman/wav2vec2-large-xlsr-53-dutch",
"uk": "Yehor/wav2vec2-xls-r-300m-uk-with-small-lm",
"pt": "jonatasgrosman/wav2vec2-large-xlsr-53-portuguese",
"ar": "jonatasgrosman/wav2vec2-large-xlsr-53-arabic",
"ru": "jonatasgrosman/wav2vec2-large-xlsr-53-russian",
"pl": "jonatasgrosman/wav2vec2-large-xlsr-53-polish",
"hu": "jonatasgrosman/wav2vec2-large-xlsr-53-hungarian",
"fi": "jonatasgrosman/wav2vec2-large-xlsr-53-finnish",
"fa": "jonatasgrosman/wav2vec2-large-xlsr-53-persian",
"el": "jonatasgrosman/wav2vec2-large-xlsr-53-greek",
"tr": "mpoyraz/wav2vec2-xls-r-300m-cv7-turkish",
"da": "saattrupdan/wav2vec2-xls-r-300m-ftspeech",
"he": "imvladikon/wav2vec2-xls-r-300m-hebrew",

[oep42]

[Purfview]

Maybe someone will implement clean wav2vec to Faster-Whisper without other stuff present in WhisperX and without dependency hell.

I also tried your Faster-Whisper standalone, which is really very fast, but it didn't produce the accurate alignment...

That's normal, Whisper is not meant to produce accurate timestamps.

I am still trying to find out if the single-word-subtitles output of Whisper-timestamped can be converted into well-aligned normal subtitles.

So, Whisper-timestamped doesn't produce normal accurately timestamped subtitles? Just separate words?

I had some problems first to download the required models myself, until I understood that with Faster-Whisper, a model is not a single file, but a folder containing four files. For downloading those models oneself, one has to know what name...

Example of the folder names is on the first page, and it's actually same as you see in the download link.
Maybe downloads failed because of your firewall.

I tried a longer video (about 1.5 hours) with Faster-Whisper standalone, but this didn't work, and gave this (incomplete) error message: "was unexpected at this time".

Post screenshot or full text when it happens.
Probably you made an error in the command syntax.

I found out that Subtitle Edit doesn't expect the models to be in Whisper-Faster\_models

Yes, currently SE doesn't have option to select models folder, it wants models in Username\.cache... to be able to select model, but those models are not used from there.
Btw, in SE transcription & timestamps are expected to be worse than using Faster-Whisper directly [same with WhisperX v3 too].

[oep42]

[Purfview]

Thanks for info.
I'll try WhisperX v2 again when I'll have time.

Is having this worse transcription and worse timestamps, still the case, when there is no checkmark near "Auto adjust timings", and/or when there is no checkmark near "Use post-processing..." in Subtitle Edit's "Audio to text" dialog box?

It's not about SE settings, it's because SE is using FFmpeg when Faster-Whisper internally is using PyAV, and results from PyAV are better for some reason.

[Purfview]

This was the error message: "An attempt was made to access a socket in a way forbidden by its access permissions". See my last comment SubtitleEdit/subtitleedit#6884 about this.

SE downloads has nothing to do with this repo.

[Purfview]

I've seen this feedback on WhisperX - "timings no better than Whisper-Faster, even worst."