What is the best way to lower H.264 decode latency?
- Thread starter Flux
- Start date
I'm not sure you necessarily can - apart from having multiple decoders and some AI to learn your channel hopping habits.
It's been many years since I looked at this (so take with a grain of salt) but, I seem to recall that, due to the extremely variable rate of encoding, the actual H.264 spec might mandate that the decoder maintain a large buffer of the undecoded stream, which will add to latency.
Further, there are 3 methods of encoding frames: I, P, & B. If you change channels, the decoder has to receive an I frame in the stream before it can produce sensible output, but you don't want too many of those in the compressed stream as they are the largest. Further, though I don't know if this is used in TV broadcasts, B frames can reference pixels of both past and future frames, which means frames may be sent out of order, again requiring quite a bit of buffering (and latency) to cope.
It's been many years since I looked at this (so take with a grain of salt) but, I seem to recall that, due to the extremely variable rate of encoding, the actual H.264 spec might mandate that the decoder maintain a large buffer of the undecoded stream, which will add to latency.
Further, there are 3 methods of encoding frames: I, P, & B. If you change channels, the decoder has to receive an I frame in the stream before it can produce sensible output, but you don't want too many of those in the compressed stream as they are the largest. Further, though I don't know if this is used in TV broadcasts, B frames can reference pixels of both past and future frames, which means frames may be sent out of order, again requiring quite a bit of buffering (and latency) to cope.
itsmydamnation
Veteran
yet the H264 encodes used on highend VC like tellepresence have no real noticeable latency. The thing about H264 is it encompasses such a large set of potential configurations and results in every metric can be very very different.
They trade off bandwidth/quality for latency.
If you produce a stream of all I frames, the latency is the time it takes to decode 1 frame, but the compression is greatly reduced. The VC apps just ensure the are enough I frames in the stream.
One of the tweaks the steaming gameplay guys do is to omit B frames, because they add an additional frame of latency.
Blazkowicz
Legend
It's nice that these trade off can be made, first time I read about using h264 for gaming and remote apps I was like, wtf?, you should send still frames only. And h264 does support sending a series of still frames (the option to have only I frames, discussed above) so it can work like MJPEG, one significant (and old) standard.
I think there has been some confusion because there are different latencies in the various systems. I assumed Flux was referring to the delay you get swapping channels, while Ismyd is concerned with the perceived end-to-end delay in the conferencing system (which won't be channel surfing).
With a fixed hardware decoder you can't do much more than encode with x.264 with low latency settings and pray.
Also keep in mind that latency isn't only a function of your ISP and their infrastructure, but also the other endpoint of your stream.
For example - one ISP may have low latency to locations in upstate New York, but may have higher latencies to the midwest simply due to the tier-1 providers that they are tied into and the backbone they just happen to be riding. Another ISP, even in the same city, may have differing latency entirely and not because they have specifically better or worse infrastructure.
For example - one ISP may have low latency to locations in upstate New York, but may have higher latencies to the midwest simply due to the tier-1 providers that they are tied into and the backbone they just happen to be riding. Another ISP, even in the same city, may have differing latency entirely and not because they have specifically better or worse infrastructure.
What if there are no hops between the server and the client. Just one direct(very high bandwidth) connection to the network.
There are no B frames,the GOP is 1, and the x264 encoder is set to "ultrafast"(requires a higher bitrate),consistant quality,1 pass, using slides.
http://x264dev.multimedia.cx/archives/249
The ASIC used in the setop box is one of these ultra low latency decoders.
http://www.jointwave-tech.com/products.html
"up to 6 ms end to end latency"
http://www.prnewswire.com/news-rele...-with-13-ms-end-to-end-latency-201632301.html
"up to 13ms of latency end to end"
There are no B frames,the GOP is 1, and the x264 encoder is set to "ultrafast"(requires a higher bitrate),consistant quality,1 pass, using slides.
http://x264dev.multimedia.cx/archives/249
The ASIC used in the setop box is one of these ultra low latency decoders.
http://www.jointwave-tech.com/products.html
"up to 6 ms end to end latency"
http://www.prnewswire.com/news-rele...-with-13-ms-end-to-end-latency-201632301.html
"up to 13ms of latency end to end"
Silent_Buddha
Legend
Over a local network? It should be less than 1 ms for just transferring bits. Applications will then add their own latency to whatever actually needs to be done.
Over the internet, good luck getting a connection with no hops.
Regards,
SB
Blazkowicz
Legend
The Nvidia Shield is one (it concentrates on fat duties as well), and the Onlive is/was one if it does/did use h264. Dunno about sound, seems the hot codec for that stuff would be Opus, it's optimized for low delay and covering the whole bitrate range from phone or worse quality to high fidelity.
Microsoft RemoteFX seems to use its own encoding stuff (they say it does "bitmaps" but not more precision apart it being a proprietary protocol with decoding doable on GPU or ASIC, and encoding done in software or with some unique hardware PCIe card)
Microsoft RemoteFX seems to use its own encoding stuff (they say it does "bitmaps" but not more precision apart it being a proprietary protocol with decoding doable on GPU or ASIC, and encoding done in software or with some unique hardware PCIe card)
