What is the best way to lower H.264 decode latency?

Flux

Regular
What is the best way to lower H.264 decode latency?
Say you have a set-top box that has a hardware decoder installed in it.
What would decrease latency to the television/monitor screen?
 

Simon F

Tea maker
Moderator
Veteran
I'm not sure you necessarily can - apart from having multiple decoders and some AI to learn your channel hopping habits.

It's been many years since I looked at this (so take with a grain of salt) but, I seem to recall that, due to the extremely variable rate of encoding, the actual H.264 spec might mandate that the decoder maintain a large buffer of the undecoded stream, which will add to latency.

Further, there are 3 methods of encoding frames: I, P, & B. If you change channels, the decoder has to receive an I frame in the stream before it can produce sensible output, but you don't want too many of those in the compressed stream as they are the largest. Further, though I don't know if this is used in TV broadcasts, B frames can reference pixels of both past and future frames, which means frames may be sent out of order, again requiring quite a bit of buffering (and latency) to cope.
 
yet the H264 encodes used on highend VC like tellepresence have no real noticeable latency. The thing about H264 is it encompasses such a large set of potential configurations and results in every metric can be very very different.
 

ERP

Veteran
yet the H264 encodes used on highend VC like tellepresence have no real noticeable latency. The thing about H264 is it encompasses such a large set of potential configurations and results in every metric can be very very different.

They trade off bandwidth/quality for latency.
If you produce a stream of all I frames, the latency is the time it takes to decode 1 frame, but the compression is greatly reduced. The VC apps just ensure the are enough I frames in the stream.

One of the tweaks the steaming gameplay guys do is to omit B frames, because they add an additional frame of latency.
 
It's nice that these trade off can be made, first time I read about using h264 for gaming and remote apps I was like, wtf?, you should send still frames only. And h264 does support sending a series of still frames (the option to have only I frames, discussed above) so it can work like MJPEG, one significant (and old) standard.
 

Rurouni

Veteran
Logically, it could also use P frames instead of I frames only to reduce bandwidth requirement without incurring additional latency.
 

Simon F

Tea maker
Moderator
Veteran
I think there has been some confusion because there are different latencies in the various systems. I assumed Flux was referring to the delay you get swapping channels, while Ismyd is concerned with the perceived end-to-end delay in the conferencing system (which won't be channel surfing).
 

MfA

Legend
What is the best way to lower H.264 decode latency?
Say you have a set-top box that has a hardware decoder installed in it.
What would decrease latency to the television/monitor screen?
With a fixed hardware decoder you can't do much more than encode with x.264 with low latency settings and pray.
 

Rurouni

Veteran
The only thing that I can think of is basically the same as when you want to lower the latency for gaming like turn off the TV post processing/use game mode. If the content is streamed through the net, then maybe try to find the lowest latency ISP in your area.
 

Albuquerque

Red-headed step child
Veteran
Also keep in mind that latency isn't only a function of your ISP and their infrastructure, but also the other endpoint of your stream.

For example - one ISP may have low latency to locations in upstate New York, but may have higher latencies to the midwest simply due to the tier-1 providers that they are tied into and the backbone they just happen to be riding. Another ISP, even in the same city, may have differing latency entirely and not because they have specifically better or worse infrastructure.
 

Flux

Regular
What if there are no hops between the server and the client. Just one direct(very high bandwidth) connection to the network.

There are no B frames,the GOP is 1, and the x264 encoder is set to "ultrafast"(requires a higher bitrate),consistant quality,1 pass, using slides.

http://x264dev.multimedia.cx/archives/249

The ASIC used in the setop box is one of these ultra low latency decoders.

http://www.jointwave-tech.com/products.html

"up to 6 ms end to end latency"

http://www.prnewswire.com/news-rele...-with-13-ms-end-to-end-latency-201632301.html

"up to 13ms of latency end to end"
 

Flux

Regular
What is the latency from one thin-client to the server?

No hops just a direct connection to the server.
 
What is the latency from one thin-client to the server?

No hops just a direct connection to the server.

Over a local network? It should be less than 1 ms for just transferring bits. Applications will then add their own latency to whatever actually needs to be done.

Over the internet, good luck getting a connection with no hops.

Regards,
SB
 

Flux

Regular
Thank you Silent Buddha.

No less than 1 ms with a direct link to the server.

Why no thin clients with x264/acc support?
 
The Nvidia Shield is one (it concentrates on fat duties as well), and the Onlive is/was one if it does/did use h264. Dunno about sound, seems the hot codec for that stuff would be Opus, it's optimized for low delay and covering the whole bitrate range from phone or worse quality to high fidelity.

Microsoft RemoteFX seems to use its own encoding stuff (they say it does "bitmaps" but not more precision apart it being a proprietary protocol with decoding doable on GPU or ASIC, and encoding done in software or with some unique hardware PCIe card)
 
Top