What is the best way to lower H.264 decode latency?

Discussion in 'PC Hardware, Software and Displays' started by Flux, Sep 6, 2013.

  1. Flux

    Regular

    Joined:
    Nov 10, 2006
    Messages:
    313
    Likes Received:
    2
    What is the best way to lower H.264 decode latency?
    Say you have a set-top box that has a hardware decoder installed in it.
    What would decrease latency to the television/monitor screen?
     
  2. Simon F

    Simon F Tea maker
    Moderator Veteran

    Joined:
    Feb 8, 2002
    Messages:
    4,560
    Likes Received:
    157
    Location:
    In the Island of Sodor, where the steam trains lie
    I'm not sure you necessarily can - apart from having multiple decoders and some AI to learn your channel hopping habits.

    It's been many years since I looked at this (so take with a grain of salt) but, I seem to recall that, due to the extremely variable rate of encoding, the actual H.264 spec might mandate that the decoder maintain a large buffer of the undecoded stream, which will add to latency.

    Further, there are 3 methods of encoding frames: I, P, & B. If you change channels, the decoder has to receive an I frame in the stream before it can produce sensible output, but you don't want too many of those in the compressed stream as they are the largest. Further, though I don't know if this is used in TV broadcasts, B frames can reference pixels of both past and future frames, which means frames may be sent out of order, again requiring quite a bit of buffering (and latency) to cope.
     
  3. itsmydamnation

    Veteran Regular

    Joined:
    Apr 29, 2007
    Messages:
    1,301
    Likes Received:
    397
    Location:
    Australia
    yet the H264 encodes used on highend VC like tellepresence have no real noticeable latency. The thing about H264 is it encompasses such a large set of potential configurations and results in every metric can be very very different.
     
  4. Davros

    Legend

    Joined:
    Jun 7, 2004
    Messages:
    15,035
    Likes Received:
    2,381
    why would you need to ?
    pretty sure the frame can be decoded within the needed time
     
  5. ERP

    ERP Moderator
    Moderator Veteran

    Joined:
    Feb 11, 2002
    Messages:
    3,669
    Likes Received:
    49
    Location:
    Redmond, WA
    They trade off bandwidth/quality for latency.
    If you produce a stream of all I frames, the latency is the time it takes to decode 1 frame, but the compression is greatly reduced. The VC apps just ensure the are enough I frames in the stream.

    One of the tweaks the steaming gameplay guys do is to omit B frames, because they add an additional frame of latency.
     
  6. Blazkowicz

    Legend Veteran

    Joined:
    Dec 24, 2004
    Messages:
    5,607
    Likes Received:
    256
    It's nice that these trade off can be made, first time I read about using h264 for gaming and remote apps I was like, wtf?, you should send still frames only. And h264 does support sending a series of still frames (the option to have only I frames, discussed above) so it can work like MJPEG, one significant (and old) standard.
     
  7. Rurouni

    Veteran

    Joined:
    Sep 30, 2008
    Messages:
    950
    Likes Received:
    216
    Logically, it could also use P frames instead of I frames only to reduce bandwidth requirement without incurring additional latency.
     
  8. Simon F

    Simon F Tea maker
    Moderator Veteran

    Joined:
    Feb 8, 2002
    Messages:
    4,560
    Likes Received:
    157
    Location:
    In the Island of Sodor, where the steam trains lie
    I think there has been some confusion because there are different latencies in the various systems. I assumed Flux was referring to the delay you get swapping channels, while Ismyd is concerned with the perceived end-to-end delay in the conferencing system (which won't be channel surfing).
     
  9. MfA

    MfA
    Legend

    Joined:
    Feb 6, 2002
    Messages:
    6,836
    Likes Received:
    481
    With a fixed hardware decoder you can't do much more than encode with x.264 with low latency settings and pray.
     
  10. Simon F

    Simon F Tea maker
    Moderator Veteran

    Joined:
    Feb 8, 2002
    Messages:
    4,560
    Likes Received:
    157
    Location:
    In the Island of Sodor, where the steam trains lie
    Surely that's out of his control? It's the broadcaster who makes that decision.
     
  11. Rurouni

    Veteran

    Joined:
    Sep 30, 2008
    Messages:
    950
    Likes Received:
    216
    The only thing that I can think of is basically the same as when you want to lower the latency for gaming like turn off the TV post processing/use game mode. If the content is streamed through the net, then maybe try to find the lowest latency ISP in your area.
     
  12. Albuquerque

    Albuquerque Red-headed step child
    Veteran

    Joined:
    Jun 17, 2004
    Messages:
    3,845
    Likes Received:
    329
    Location:
    35.1415,-90.056
    Also keep in mind that latency isn't only a function of your ISP and their infrastructure, but also the other endpoint of your stream.

    For example - one ISP may have low latency to locations in upstate New York, but may have higher latencies to the midwest simply due to the tier-1 providers that they are tied into and the backbone they just happen to be riding. Another ISP, even in the same city, may have differing latency entirely and not because they have specifically better or worse infrastructure.
     
  13. Flux

    Regular

    Joined:
    Nov 10, 2006
    Messages:
    313
    Likes Received:
    2
    What if there are no hops between the server and the client. Just one direct(very high bandwidth) connection to the network.

    There are no B frames,the GOP is 1, and the x264 encoder is set to "ultrafast"(requires a higher bitrate),consistant quality,1 pass, using slides.

    http://x264dev.multimedia.cx/archives/249

    The ASIC used in the setop box is one of these ultra low latency decoders.

    http://www.jointwave-tech.com/products.html

    "up to 6 ms end to end latency"

    http://www.prnewswire.com/news-rele...-with-13-ms-end-to-end-latency-201632301.html

    "up to 13ms of latency end to end"
     
  14. Flux

    Regular

    Joined:
    Nov 10, 2006
    Messages:
    313
    Likes Received:
    2
    What is the latency from one thin-client to the server?

    No hops just a direct connection to the server.
     
  15. Silent_Buddha

    Legend

    Joined:
    Mar 13, 2007
    Messages:
    16,342
    Likes Received:
    5,317
    Over a local network? It should be less than 1 ms for just transferring bits. Applications will then add their own latency to whatever actually needs to be done.

    Over the internet, good luck getting a connection with no hops.

    Regards,
    SB
     
  16. Flux

    Regular

    Joined:
    Nov 10, 2006
    Messages:
    313
    Likes Received:
    2
    Thank you Silent Buddha.

    No less than 1 ms with a direct link to the server.

    Why no thin clients with x264/acc support?
     
  17. Blazkowicz

    Legend Veteran

    Joined:
    Dec 24, 2004
    Messages:
    5,607
    Likes Received:
    256
    The Nvidia Shield is one (it concentrates on fat duties as well), and the Onlive is/was one if it does/did use h264. Dunno about sound, seems the hot codec for that stuff would be Opus, it's optimized for low delay and covering the whole bitrate range from phone or worse quality to high fidelity.

    Microsoft RemoteFX seems to use its own encoding stuff (they say it does "bitmaps" but not more precision apart it being a proprietary protocol with decoding doable on GPU or ASIC, and encoding done in software or with some unique hardware PCIe card)
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...