Audio capture and streaming app for webOS 5/6
Captures the TV's audio and sends it out over several transports. The
primary one is HyperHDR: RTP/L16 to a host-side loopback device, since
HyperHDR has no network audio input of its own. A second route renders
the spectrum on the TV and sends FlatBuffers images to port 19400
instead, for setups where touching the host's sound config is not an
option.
native/ the service: capture backends (PulseAudio, ALSA, exec,
test tone, all dlopen-based), DSP, and one file per sink
frontend/ D-pad driven UI at a fixed 1920x1080
servicefiles/ native service manifest plus the boot script
host/ RTP receiver and the loopback installer for the HyperHDR
machine
tools/ build/package, asset generation, Homebrew Channel
manifest, on-TV probe
test/ host-side suites: FlatBuffers and RTP verified against
real decoders, the engine end to end, the page in jsdom
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Rene Kievits
@@ -0,0 +1,163 @@
|
||||
# Connecting to HyperHDR
|
||||
|
||||
HyperHDR's music effects read a **local capture device**. There is no network
|
||||
audio input to send to, no API to push samples into. Everything below is a way
|
||||
of working around that.
|
||||
|
||||
Three routes, in the order you should try them.
|
||||
|
||||
---
|
||||
|
||||
## 1. RTP audio into a loopback device (recommended)
|
||||
|
||||
The TV sends RTP/L16 to the HyperHDR machine; a receiver there plays it into a
|
||||
loopback; HyperHDR captures the other end of that loopback. HyperHDR sees a
|
||||
perfectly ordinary sound card and does its own analysis, so every effect works
|
||||
exactly as it would with a real input.
|
||||
|
||||
```
|
||||
TV ──RTP/L16 udp/5004──► lgtv-audiocap-receiver.py ──► hw:Loopback,0,0
|
||||
║ snd-aloop
|
||||
HyperHDR ◄──── hw:Loopback,1,0
|
||||
```
|
||||
|
||||
### On the HyperHDR machine
|
||||
|
||||
```sh
|
||||
git clone <this repo> && cd lgtv-audio-cap
|
||||
sudo ./host/install-loopback.sh --install-service
|
||||
```
|
||||
|
||||
That loads `snd-aloop` (persisting it across reboots), keeps PulseAudio's hands
|
||||
off the loopback card, installs the receiver into `/usr/local/bin` and starts it
|
||||
as a systemd unit. It finishes by printing the exact device name to give
|
||||
HyperHDR.
|
||||
|
||||
To do it by hand instead:
|
||||
|
||||
```sh
|
||||
sudo modprobe snd-aloop index=10 pcm_substreams=1 id=Loopback
|
||||
./host/lgtv-audiocap-receiver.py --output aplay --device hw:Loopback,0,0
|
||||
```
|
||||
|
||||
### On the TV
|
||||
|
||||
*Outputs → HyperHDR audio (RTP/L16)*
|
||||
|
||||
| Setting | Value |
|
||||
| --- | --- |
|
||||
| Receiver address | the HyperHDR machine's IP |
|
||||
| UDP port | 5004 |
|
||||
| Multicast | off |
|
||||
| Announce over SAP | on (harmless, and needed for route 2) |
|
||||
|
||||
Press **Start**.
|
||||
|
||||
### In HyperHDR
|
||||
|
||||
Settings → *Sound capture* (in newer builds; older ones put it under the music
|
||||
effect itself) → input device `hw:Loopback,1,0`, then choose a music effect.
|
||||
|
||||
### Checking it
|
||||
|
||||
```sh
|
||||
# is anything arriving at all?
|
||||
./host/lgtv-audiocap-receiver.py --port 5004 --output - | \
|
||||
aplay -f S16_LE -r 48000 -c 2 -
|
||||
```
|
||||
|
||||
The receiver prints a line every 30 seconds with packet, loss and restart
|
||||
counts. Losses in the low hundreds over hours are normal on Wi-Fi; a steady
|
||||
stream of them means the TV's Wi-Fi is the bottleneck and the set really wants
|
||||
Ethernet.
|
||||
|
||||
---
|
||||
|
||||
## 2. RTP straight into PulseAudio, nothing installed
|
||||
|
||||
If the HyperHDR machine runs PulseAudio or PipeWire and HyperHDR can reach it
|
||||
through the ALSA `pulse` device, you do not need the receiver at all. The TV
|
||||
announces the stream over SAP and PulseAudio builds a source from it.
|
||||
|
||||
```sh
|
||||
pactl load-module module-rtp-recv sap_address=224.0.0.56
|
||||
```
|
||||
|
||||
With *Announce over SAP* enabled on the TV, a source called something like
|
||||
`rtp_recv.LG TV Audio Cap` appears within five seconds. Point HyperHDR at its
|
||||
monitor.
|
||||
|
||||
This is the least code, but it is also the least predictable: PulseAudio's RTP
|
||||
receiver has no jitter buffer worth the name, and PipeWire's compatibility layer
|
||||
does not always implement the module. Treat it as a nice surprise if it works.
|
||||
|
||||
For unicast rather than SAP discovery, turn *Announce over SAP* off and load:
|
||||
|
||||
```sh
|
||||
pactl load-module module-rtp-recv sap_address=0.0.0.0 port=5004
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. The TV does the visualising
|
||||
|
||||
No host software, no sound device. The TV analyses the audio, renders a small
|
||||
image and sends it to HyperHDR's Flatbuffers port, the same way a
|
||||
`hyperion-remote` or a screen grabber would.
|
||||
|
||||
*Outputs → HyperHDR visualiser*
|
||||
|
||||
| Setting | Value |
|
||||
| --- | --- |
|
||||
| HyperHDR address | the HyperHDR machine's IP |
|
||||
| Flatbuffers port | 19400 |
|
||||
| Style | Spectrum, Level bar or Pulse |
|
||||
| Priority | 150 (lower numbers win in HyperHDR) |
|
||||
|
||||
In HyperHDR, make sure the Flatbuffers server is enabled (Settings → Network
|
||||
Services → Flatbuffers server, default port 19400).
|
||||
|
||||
What you give up: HyperHDR's own effects, colour calibration on the audio path,
|
||||
and any hope of the lights matching an effect you have configured elsewhere. The
|
||||
TV decides what the lights show. What you gain: it works in about a minute.
|
||||
|
||||
The three styles:
|
||||
|
||||
- **Spectrum** — 16 bands across the image, hue by frequency.
|
||||
- **Level bar** — one bar that tracks the overall level.
|
||||
- **Pulse** — the whole image flashes with the beat.
|
||||
|
||||
`saturation` and `minBrightness` shape the output; `minBrightness: 0` lets the
|
||||
lights go fully dark between beats, which looks dramatic and slightly broken.
|
||||
|
||||
---
|
||||
|
||||
## Which one to use
|
||||
|
||||
| | Route 1 | Route 2 | Route 3 |
|
||||
| --- | --- | --- | --- |
|
||||
| Host software | receiver + loopback | none | none |
|
||||
| HyperHDR effects | all of them | all of them | none, the TV renders |
|
||||
| Latency | ~100 ms | ~100 ms, less stable | ~40 ms |
|
||||
| Robustness | good | depends on your PulseAudio | good |
|
||||
| Setup time | 10 minutes | 2 minutes if it works | 1 minute |
|
||||
|
||||
Route 1 unless you have a reason.
|
||||
|
||||
---
|
||||
|
||||
## Latency
|
||||
|
||||
Roughly, end to end on route 1:
|
||||
|
||||
| Stage | Typical |
|
||||
| --- | --- |
|
||||
| TV capture block | 11 ms (512 frames at 48 kHz) |
|
||||
| Network | 1–5 ms wired, 5–40 ms Wi-Fi |
|
||||
| Receiver prebuffer | 60 ms, `--prebuffer-ms` |
|
||||
| Playback buffer | 80 ms, `--latency-ms` |
|
||||
| HyperHDR's own analysis | 20–50 ms |
|
||||
|
||||
Around 150–200 ms in total, which for ambient lighting is imperceptible. If you
|
||||
want it tighter, lower `--latency-ms` and `--prebuffer-ms` until the audio
|
||||
starts crackling, then go back up one step.
|
||||
Reference in New Issue
Block a user