141 linhas
3.0 KiB
Plaintext
141 linhas
3.0 KiB
Plaintext
# Whisper.cpp - Speech-to-Text with Vulkan GPU
|
|
|
|
High-performance Speech-to-Text using OpenAI's Whisper model with Vulkan GPU acceleration.
|
|
|
|
## Quick Start
|
|
|
|
### 1. Build
|
|
|
|
```bash
|
|
cd Services/Whisper
|
|
./build-container.sh
|
|
```
|
|
|
|
Done! The image is ready.
|
|
|
|
### 2. Run
|
|
|
|
```bash
|
|
podman run --device /dev/dri/renderD128 -p 8080:8080 \
|
|
-v /srv/whisper/models:/app/models \
|
|
whisper:vulkan-amd64
|
|
```
|
|
|
|
Default model is `small` (downloads on first start).
|
|
|
|
**Change model without rebuilding:**
|
|
```bash
|
|
podman run --device /dev/dri/renderD128 -p 8080:8080 \
|
|
-v /srv/whisper/models:/app/models \
|
|
-e WHISPER_MODEL_NAME=medium \
|
|
whisper:vulkan-amd64
|
|
```
|
|
|
|
### 3. Transcribe
|
|
|
|
**JSON output:**
|
|
```bash
|
|
curl -F "file=@audio.wav" http://localhost:8080/inference > result.json
|
|
```
|
|
|
|
**Extract text to file:**
|
|
```bash
|
|
curl -F "file=@audio.wav" http://localhost:8080/inference | jq -r '.text' > transcript.txt
|
|
```
|
|
|
|
**Supported formats:** .wav, .mp3, .ogg, .flac, .opus
|
|
|
|
## Available Models
|
|
|
|
Set with: `WHISPER_MODEL_NAME=<model>`
|
|
|
|
| Model | Size | Speed | Memory | Default |
|
|
|-------|------|-------|--------|---------|
|
|
| small | 466 MB | Good | ~1.1 GB | ✓ |for file in *.opus; do
|
|
ffmpeg -i "$file" -acodec pcm_s16le "${file%.opus}.wav" -y
|
|
done
|
|
| medium | 775 MB | Better | ~1.2 GB | |
|
|
| large-v3 | 2.9 GB | Best | ~3.9 GB | |
|
|
|
|
Models auto-download on first use.
|
|
|
|
### Change Model at Runtime
|
|
|
|
```bash
|
|
# Run with medium instead of small
|
|
podman run -e WHISPER_MODEL_NAME=medium whisper:vulkan-amd64
|
|
|
|
# Change in systemd service (edit whisper.container)
|
|
Environment=WHISPER_MODEL_NAME=medium
|
|
```
|
|
|
|
## Basic Commands
|
|
|
|
```bash
|
|
# Transcribe (uses configured model)
|
|
podman exec whisper whisper-cli -m /app/models/ggml-small.bin -f audio.wav
|
|
|
|
# Benchmark
|
|
podman exec whisper whisper-bench -m /app/models/ggml-small.bin
|
|
|
|
# Check logs
|
|
podman logs whisper
|
|
```
|
|
|
|
## Systemd Service
|
|
|
|
```bash
|
|
# Install
|
|
podman container runlabel install -n whisper whisper.container localhost/whisper:vulkan-amd64
|
|
|
|
# Enable and start
|
|
systemctl --user enable --now whisper
|
|
|
|
# Check status
|
|
systemctl --user status whisper
|
|
```
|
|
|
|
## Troubleshooting
|
|
|
|
**GPU not detected:**
|
|
```bash
|
|
podman run --device /dev/dri/renderD128 --rm whisper:vulkan-amd64 vulkaninfo
|
|
```
|
|
|
|
**WSL2 (GPU device and Vulkan drivers):**
|
|
|
|
WSL2 uses different GPU device paths and Vulkan drivers come from WSLg. You may need to:
|
|
|
|
```bash
|
|
# Option 1: Mount WSL GPU libraries
|
|
podman run --device /dev/dri/dgx \
|
|
-v /usr/lib/wsl:/usr/lib/wsl:ro \
|
|
-p 8080:8080 \
|
|
-v /srv/whisper/models:/app/models \
|
|
whisper:vulkan-amd64
|
|
|
|
# Option 2: If Option 1 fails, also mount vulkan drivers
|
|
podman run --device /dev/dri/dgx \
|
|
-v /usr/lib/wsl:/usr/lib/wsl:ro \
|
|
-v /usr/share/vulkan:/usr/share/vulkan:ro \
|
|
-p 8080:8080 \
|
|
-v /srv/whisper/models:/app/models \
|
|
-e VK_DRIVER_FILES=/usr/share/vulkan/icd.d/icd.json \
|
|
whisper:vulkan-amd64
|
|
|
|
# Find the correct GPU device:
|
|
ls /dev/dri/
|
|
# Usually: /dev/dri/dgx for GPU
|
|
```
|
|
|
|
**Out of memory:**
|
|
Use `medium-q5` instead of `large-v3`
|
|
|
|
## References
|
|
|
|
- [whisper.cpp](https://github.com/ggml-org/whisper.cpp)
|
|
- [Vulkan](https://www.khronos.org/vulkan/)
|
|
|
|
|
|
|