lgtv_audio_cap/docs/hyperhdr.md

# Connecting to HyperHDR

HyperHDR's music effects read a **local capture device**. There is no network
audio input to send to, no API to push samples into. Everything below is a way
of working around that.

Three routes, in the order you should try them.

---

## 1. RTP audio into a loopback device (recommended)

The TV sends RTP/L16 to the HyperHDR machine; a receiver there plays it into a
loopback; HyperHDR captures the other end of that loopback. HyperHDR sees a
perfectly ordinary sound card and does its own analysis, so every effect works
exactly as it would with a real input.

```
TV ──RTP/L16 udp/5004──► lgtv-audiocap-receiver.py ──► hw:Loopback,0,0
                                                             ║ snd-aloop
                                        HyperHDR ◄──── hw:Loopback,1,0
```

### On the HyperHDR machine

```sh
git clone <this repo> && cd lgtv-audio-cap
sudo ./host/install-loopback.sh --install-service
```

That loads `snd-aloop` (persisting it across reboots), keeps PulseAudio's hands
off the loopback card, installs the receiver into `/usr/local/bin` and starts it
as a systemd unit. It finishes by printing the exact device name to give
HyperHDR.

To do it by hand instead:

```sh
sudo modprobe snd-aloop index=10 pcm_substreams=1 id=Loopback
./host/lgtv-audiocap-receiver.py --output aplay --device hw:Loopback,0,0
```

### On Unraid

Unraid boots from a read-only USB image, so nothing here can be "just a
systemd service" — the loopback and the receiver need to be split into the
one part that genuinely needs the bare-metal kernel and the part that doesn't.

**The loopback (bare metal):** install
[`unraid/lgtv-audiocap-loopback.plg`](../unraid/lgtv-audiocap-loopback.plg) —
*Plugins → Install Plugin*, paste the raw URL to that file. It loads
`snd-aloop` immediately and adds one line to `/boot/config/go` so it survives
a reboot; *Plugins → Uninstall* removes exactly that line and nothing else.

**The receiver (a normal container):** build
[`docker/Dockerfile`](../docker/Dockerfile) and add it like any other Unraid
container — *Docker → Add Container*:

| Setting | Value |
| --- | --- |
| Repository | your image, e.g. `192.168.0.4:5000/lgtv-audiocap-receiver` |
| Network Type | Bridge (or Host, either works — it only ever listens on one UDP port) |
| Port | `5004` UDP → `5004` |
| Extra Parameters | `--device /dev/snd:/dev/snd` |

It's entirely configured through environment variables — see
[`docker/entrypoint.sh`](../docker/entrypoint.sh) for the full list
(`PORT`, `DEVICE`, `RATE`, `CHANNELS`, `LATENCY_MS`, …). The default `DEVICE`
is already `hw:Loopback,0,0`, so nothing needs setting for the common case.

Point the **HyperHDR container** at the loopback the same way: add
`--device /dev/snd:/dev/snd` to its extra parameters too, then use
`hw:Loopback,1,0` in its Sound Capture settings. Both containers reach the
same host kernel device, so no networking between them is needed for this
part — only the TV needs to know the host's IP, for the RTP stream itself.

### On the TV

*Outputs → HyperHDR audio (RTP/L16)*

| Setting | Value |
| --- | --- |
| Receiver address | the HyperHDR machine's IP |
| UDP port | 5004 |
| Multicast | off |
| Announce over SAP | on (harmless, and needed for route 2) |

Press **Start**.

### In HyperHDR

Settings → *Sound capture* (in newer builds; older ones put it under the music
effect itself) → input device `hw:Loopback,1,0`, then choose a music effect.

### Checking it

```sh
# is anything arriving at all?
./host/lgtv-audiocap-receiver.py --port 5004 --output - | \
    aplay -f S16_LE -r 48000 -c 2 -
```

The receiver prints a line every 30 seconds with packet, loss and restart
counts. Losses in the low hundreds over hours are normal on Wi-Fi; a steady
stream of them means the TV's Wi-Fi is the bottleneck and the set really wants
Ethernet.

---

## 2. RTP straight into PulseAudio, nothing installed

If the HyperHDR machine runs PulseAudio or PipeWire and HyperHDR can reach it
through the ALSA `pulse` device, you do not need the receiver at all. The TV
announces the stream over SAP and PulseAudio builds a source from it.

```sh
pactl load-module module-rtp-recv sap_address=224.0.0.56
```

With *Announce over SAP* enabled on the TV, a source called something like
`rtp_recv.LG TV Audio Cap` appears within five seconds. Point HyperHDR at its
monitor.

This is the least code, but it is also the least predictable: PulseAudio's RTP
receiver has no jitter buffer worth the name, and PipeWire's compatibility layer
does not always implement the module. Treat it as a nice surprise if it works.

For unicast rather than SAP discovery, turn *Announce over SAP* off and load:

```sh
pactl load-module module-rtp-recv sap_address=0.0.0.0 port=5004
```

---

## 3. The TV does the visualising

No host software, no sound device. The TV analyses the audio, renders a small
image and sends it to HyperHDR's Flatbuffers port, the same way a
`hyperion-remote` or a screen grabber would.

*Outputs → HyperHDR visualiser*

| Setting | Value |
| --- | --- |
| HyperHDR address | the HyperHDR machine's IP |
| Flatbuffers port | 19400 |
| Style | Spectrum, Level bar or Pulse |
| Priority | 150 (lower numbers win in HyperHDR) |

In HyperHDR, make sure the Flatbuffers server is enabled (Settings → Network
Services → Flatbuffers server, default port 19400).

What you give up: HyperHDR's own effects, colour calibration on the audio path,
and any hope of the lights matching an effect you have configured elsewhere. The
TV decides what the lights show. What you gain: it works in about a minute.

The three styles:

- **Spectrum** — 16 bands across the image, hue by frequency.
- **Level bar** — one bar that tracks the overall level.
- **Pulse** — the whole image flashes with the beat.

`saturation` and `minBrightness` shape the output; `minBrightness: 0` lets the
lights go fully dark between beats, which looks dramatic and slightly broken.

---

## 4. Keep your grabber's colour, only pulse the brightness

For an ambilight-style setup that already has a real colour source — a
screen grabber, a USB capture card, a webOS capture app like piccap — routes
1–3 all have the same problem: they compete for HyperHDR's priority and
*replace* that colour with something audio-derived. This route doesn't touch
colour at all.

HyperHDR has a JSON-RPC `adjustment` command that sets output brightness
(0-100) as a post-processing step, applied on top of whatever priority is
currently active. This sink sends nothing but that: no image, no priority
registration, so the grabber (or piccap, or whatever else) keeps deciding
hue and this only turns the result up and down with the sound.

```
TV ──RTP or local──► audiocap-service ──JSON-RPC "adjustment"──► HyperHDR
                                                                   (still showing
                                                                    the grabber's colour)
```

*Outputs → HyperHDR brightness (JSON-RPC)*

| Setting | Value |
| --- | --- |
| HyperHDR address | the HyperHDR machine's IP |
| JSON-RPC port | 19444 (HyperHDR's classic control port — not 8090, the web UI; not 19400, Flatbuffers) |
| Follows | Average level (steadier) or Peak level (punchier) |
| Minimum brightness | `0`-`100`; applied during quiet parts |
| Maximum brightness | `0`-`100`; applied at full level. HyperHDR does not go above 100 |

Needs a working capture source the same as every other route — see the top
of this document for picking one. On close, the sink resets brightness to
`100` rather than leaving the LEDs stuck at whatever it last sent.

The field is `brightness`, confirmed by watching it round-trip through
`serverinfo` and the LEDs visibly respond on a real HyperHDR/Docker
instance. HyperHDR's current `schema-adjustment.json` documents a
`scaleOutput` float (0.0-2.0) instead, which looked like the obvious choice
and is what this sink sent originally — it had no visible or server-reported
effect on that same instance. If a future HyperHDR version drops
`brightness`, this needs re-verifying the same way, not just re-reading the
schema.

---

## Which one to use

| | Route 1 | Route 2 | Route 3 | Route 4 |
| --- | --- | --- | --- | --- |
| Host software | receiver + loopback | none | none | none |
| HyperHDR effects | all of them | all of them | none, the TV renders | your existing grabber, untouched |
| Colour source | HyperHDR's built-in audio effect | HyperHDR's built-in audio effect | this app's synthetic spectrum | your grabber — this only adjusts brightness |
| Latency | ~100 ms | ~100 ms, less stable | ~40 ms | ~50 ms |
| Robustness | good | depends on your PulseAudio | good | good |
| Setup time | 10 minutes | 2 minutes if it works | 1 minute | 1 minute |

Route 1 for HyperHDR's own audio effects. Route 4 if you already have a
grabber and just want it to breathe with the sound instead of being replaced.

---

## Latency

Roughly, end to end on route 1:

| Stage | Typical |
| --- | --- |
| TV capture block | 11 ms (512 frames at 48 kHz) |
| Network | 1–5 ms wired, 5–40 ms Wi-Fi |
| Receiver prebuffer | 60 ms, `--prebuffer-ms` |
| Playback buffer | 80 ms, `--latency-ms` |
| HyperHDR's own analysis | 20–50 ms |

Around 150–200 ms in total, which for ambient lighting is imperceptible. If you
want it tighter, lower `--latency-ms` and `--prebuffer-ms` until the audio
starts crackling, then go back up one step.