Next Generation Hardware Speculation with a Technical Spin [post GDC 2020] [XBSX, PS5]

Discussion in 'Console Technology' started by Proelite, Mar 16, 2020.

  1. AzBat

    AzBat Agent of the Bat
    Legend Veteran

    Joined:
    Apr 1, 2002
    Messages:
    6,519
    Likes Received:
    2,602
    Location:
    Alma, AR
    Any link to this AMD VRS patent?

    Tommy McClain
     
  2. function

    function None functional
    Legend Veteran

    Joined:
    Mar 27, 2003
    Messages:
    5,270
    Likes Received:
    2,599
    Location:
    Wrong thread
    I've definitely raised that question here and discussed it. When we knew that MS were using VRS incorporating their own patents, and that Sony were silent on VRS (as they still are) people were saying that Sony must have it because it's in PC RDNA 2 parts coming later this year. But that's not necessarily the case.

    I said it was entirely plausible that AMD had an agreement with MS and that they could be sharing an implementation. It could make sense from an engineering perspective, and it would fit with the timescale of these products coming to market.

    If AMD has their own patent they could change their implementation for RDNA 3 or whatever suits their timeline.

    MS clearly wanted VRS for XSX, and they want it to be adopted as part of DX12U. AMD don't want to be releasing RDNA 2 for PC missing notable features that Nvidia have had for more than two years, especially now they're being pushed as part of the latest DX12 spec. MS want VRS in Xbox and on desktop, and so do AMD. I expect they could come to a mutually beneficial agreement.

    Besides, any licensing fee would have to be large to outweigh the hit to competitiveness against Nividia and Intel (a ~15% performance feature is not insignificant), and it might even be in MS's interests to waive a fee altogether for this particular generation of PC GPUs - especially if Sony can't have it.

    The question of "why would Sony disable VRS when it's already in RDNA 2" might be answerable with "Sony's RNDA options never included it!" :no:
     
    PSman1700 likes this.
  3. function

    function None functional
    Legend Veteran

    Joined:
    Mar 27, 2003
    Messages:
    5,270
    Likes Received:
    2,599
    Location:
    Wrong thread
    https://hexus.net/tech/news/graphics/128066-amd-navi-gpus-might-use-variable-rate-shading-tech/

    Filed in 2017, apparently!

    Edit: can anyone work out if this is tier 1 or tier 2? I can't. Maybe it covers both ...:???:

    Edit 2: Erm ....?

     
    #3123 function, Jul 1, 2020
    Last edited: Jul 1, 2020
    AzBat likes this.
  4. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    10,448
    Likes Received:
    10,119
    Location:
    The North
    on XSX low latency.
    Wondering now if this is related:
    from HAGS background:
    hmmm.. all this time I thought that was normal to be N and N-1. Going to have to think on this more.

     
    pjbliverpool likes this.
  5. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    10,448
    Likes Received:
    10,119
    Location:
    The North
    http://www.freepatentsonline.com/y2019/0066371.html

    Actual patent above.

    It doesn't read like it's Tier 2 to be honest.
    Tier 1
    • Shading rate can only be specified on a per-draw-basis; nothing more granular than that
    • Shading rate applies uniformly to what is drawn independently of where it lies within the rendertarget
    • Use of 1x2, programmable sample positions, or conservative rasterization may cause fall-back into fine shading
    Tier 2
    • Shading rate can be specified on a per-draw-basis, as in Tier 1. It can also be specified by a combination of per-draw-basis, and of:
      • Semantic from the per-provoking-vertex, and
      • a screenspace image
    • Shading rates from the three sources are combined using a set of combiners
    • Screen space image tile size is 16x16 or smaller
    • Shading rate requested by the app is guaranteed to be delivered exactly (for precision of temporal and other reconstruction filters)
    • SV_ShadingRate PS input is supported
    • The per-provoking vertex rate, also referred to here as a per-primitive rate, is valid when one viewport is used and SV_ViewportIndex is not written to.
    • The per-provoking vertex rate, also referred to as a per-primitive rate, can be used with more than one viewport if the SupportsPerVertexShadingRateWithMultipleViewports cap is marked true. Additionally, in that case, it can be used when SV_ViewportIndex is written to.

    Just to be clear, I don't know what i"m doing. Reading patents in a head scratcher.
    But it seems to me, comparing the two, the MS Patent at each stage of the unified shader pipeline, has the option to take in shading rate parameters or output them.

    Compared to AMDs patent, which is also runs VRS through the unified shader pipeline, it doesn't seem to indicate that.

    If you look at the Tier 2 highlights, it does appear that, yes, some form of supporting optional shading rate parameters on different stages.

    MS Patent
    Is here https://patents.google.com/patent/US20180047203A1/en

    As I review this, and the words 'Hololens' pop up in this patent, it would appear that MS has been working on their own from of VRS for some time in association with trying to extract as much shader power as possible while reducing the amount of power required to do it.

    That research got them to this point, and instead of going with AMDs solution, their solution was better (the Hololens one) and they plopped it here.

    That may imply, Hololens and any other VR type device that they've been working on with foveated rendering in mind (and thus VRS), could be driven by in this case, XSX. For context, Hololens 3 is supposed to launch with foveated rendering.

     
    #3125 iroboto, Jul 1, 2020
    Last edited: Jul 1, 2020
  6. Ronaldo8

    Newcomer

    Joined:
    May 18, 2020
    Messages:
    166
    Likes Received:
    149
  7. DxCore

    Newcomer

    Joined:
    Jun 22, 2020
    Messages:
    12
    Likes Received:
    5
    Nvidia / Intel / Qualcomm / AMD / MS all have VRS patents .. and roughly similar timeframes .. And it probably came about from a usergroup collaboration between them all .. There were even presentations from certain graphics folks calling out the different OEM VRS implementations making it sound like a entire group collaboration
     
  8. DxCore

    Newcomer

    Joined:
    Jun 22, 2020
    Messages:
    12
    Likes Received:
    5
    this was one such graphics team presentation , it was in the ancillary slides at the end of the presentation ...

    upload_2020-7-1_15-11-21.png
     
    pjbliverpool likes this.
  9. Ronaldo8

    Newcomer

    Joined:
    May 18, 2020
    Messages:
    166
    Likes Received:
    149
    The MS patent only describes how the shading rate is generated for a fragment of a primitive, to be used by a GPU "configured for variable pixel shading". How the variable shading rates are actually used in the GPU configured for variable pixel shading" and what this "configuration" actually consist of is described in an all encompassing patent from the inventors involved in the variable shading patent (both patents filed simultaneously): "Multiple Shader Processes in graphics processing" (https://patentimages.storage.googleapis.com/1d/60/ab/e8498e7a9089d4/US20180232936A1.pdf)

    Of particular interest here is Fig 7. describing how a particular pixel shader can be selected from a plurality of pixel shaders to shade a particular fragment from a plurality of fragments of a primitive using configurable parameters like a variable shading rate.
    MS VRS solution seems to cover both software and hardware.
     
  10. Ronaldo8

    Newcomer

    Joined:
    May 18, 2020
    Messages:
    166
    Likes Received:
    149
    The DX12 API is hardware-implementation agnostic.
     
  11. DSoup

    DSoup meh
    Legend Veteran Subscriber

    Joined:
    Nov 23, 2007
    Messages:
    12,477
    Likes Received:
    7,724
    Location:
    London, UK
    Yup. Patents are currency in the tech world. You either want to license the right to use patented technology or it's trade use of your patents for use of somebody else's.
     
  12. jayco

    Veteran Regular

    Joined:
    Nov 18, 2006
    Messages:
    1,268
    Likes Received:
    569
    What about this patent from SIE/Cerny?

    upload_2020-7-1_16-34-16.png

    https://patents.google.com/patent/US10614549B2/en
     
  13. Shifty Geezer

    Shifty Geezer uber-Troll!
    Moderator Legend

    Joined:
    Dec 7, 2004
    Messages:
    43,488
    Likes Received:
    15,941
    Location:
    Under my bridge
    I'm afraid my reading of Patentese failed in this case:

    1. A method for graphics processing with a graphics processing system having a graphics processing unit coupled to a display device, comprising: receiving metadata specifying an active sample configuration for a particular region of a screen of the display device among a plurality of regions of the screen, wherein the metadata specifies different active sample configurations for regions of the screen that have different resolutions; receiving pixel data for one or more pixels of an image in the particular region, wherein the pixel data specifies the same number of color samples for each pixel; wherein the number of color samples for each pixel specified by the pixel data is the same over an entire surface of the screen, and for each pixel in the particular region that is covered by a primitive, invoking a pixel shader only for color samples for the pixel specified to be active samples by the active sample configuration, wherein pixel shading computations of the pixel shader super-sample each pixel by virtue of being invoked for each active color sample of each pixel in the particular region that is covered by the primitive.​

    Got as far as it being relevant to conditional rendering quality specified by a display device, which would suit a VR headset and foveated rendering.
     
  14. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    10,448
    Likes Received:
    10,119
    Location:
    The North
    no failure there. it's exhausting. I hate patent diving.

    This patent as well as the VRS one from MS, both related and driven by the research from the Hololens team. They may have found its usage as being useful elsewhere as it really assists with reducing the power footprint as much as possible
     
    #3134 iroboto, Jul 1, 2020
    Last edited: Jul 1, 2020
  15. Shifty Geezer

    Shifty Geezer uber-Troll!
    Moderator Legend

    Joined:
    Dec 7, 2004
    Messages:
    43,488
    Likes Received:
    15,941
    Location:
    Under my bridge
    I lose it from this point:

    wherein the number of color samples for each pixel specified by the pixel data is the same over an entire surface of the screen, and for each pixel in the particular region that is covered by a primitive, invoking a pixel shader only for color samples for the pixel specified to be active samples by the active sample configuration, wherein pixel shading computations of the pixel shader super-sample each pixel by virtue of being invoked for each active color sample of each pixel in the particular region that is covered by the primitive.

    On the one hand, it seems like selective sampling based on an active sampling configuration, but then they mention super-sampling. ¯\_(ツ)_/¯
     
  16. Ronaldo8

    Newcomer

    Joined:
    May 18, 2020
    Messages:
    166
    Likes Received:
    149
    This patent is for the reduction of pixel density at the periphery of an HMD screen to decrease resolution of areas under peripheral vision and increase the resolution of areas under central vision. Quoting relevant part of the patent:

    "Another way of looking at this situation is shown in FIG. 2C, in which the screen 102 has been divided into rectangles of approximately equal “importance” in terms of pixels per unit solid angle subtended. Each rectangle makes roughly the same contribution to the final image as seen through the display. One can see how the planar projection distorts the importance of edge rectangles 202 and corner rectangles 203. In fact, the corner rectangles 203 might make less of a contribution to the center rectangles due to the display optics, which may choose to make the visual density of pixels (as expressed as pixels per solid angle) higher towards the center of the display."

    The only thing it has in common with VRS is that it enables variance of resolution. However, VRS achieves the latter by varying shading rate on a per primitive basis and is, instead of being tied to screen area, completely arbitrary insofar as it comes to calculating the shading rate parameter and applying it (at least in MS implementation of it). Again, quoting from this very important patent from MS (https://patentimages.storage.googleapis.com/1d/60/ab/e8498e7a9089d4/US20180232936A1.pdf):

    "In one example, a mesh shader (which may be part of a rasterizer stage, as described further herein) may operate to execute one or more thread vectors, each of which can include a plurality of lanes (e.g., threads) for independent or parallel execution (e.g., 64 lanes in some examples). In this example, the mesh shader may launch a pixel shader to operate on each of the plurality of lanes to provide substantially simultaneous shading of a plurality of pixels of the primitive, where the pixel shader can, in each lane, execute (e.g., concurrently) the same instructions for shading different sets of one or more pixels.

    The mesh shader, in an example, may be capable of providing different pixel shader parameter values for portions of a given primitive. For example, the different pixel shader parameter values may include a variable rate shading parameter, such that different shading rates (e.g., 1 pixel per pixel shader thread, 2 pixels per pixel shader thread, 4 pixels per pixel shader thread, etc.) can be applied for different portions of a given primitive. In another example, the different pixel shader parameter values may include different stencil values from a stencil buffer that can be used to determine pixel values, etc.
    "

    Shading rate parameter can be calculated using a temporal reprojection transform (described here: https://patentimages.storage.googleapis.com/31/f4/6b/42fca3322ccda9/US20190005714A1.pdf), a metric over screen space (Apple patent) or any other clever algorithm a developer will come up with. The ability to invoke pixel shaders and to vary shading rate on a per primitive basis enables an interesting form of deffered rendering that aims to trump checkerboard rendering:
    https://patentimages.storage.googleapis.com/6c/82/9e/652053ef900d5f/US20190005713A1.pdf.
     
  17. Ronaldo8

    Newcomer

    Joined:
    May 18, 2020
    Messages:
    166
    Likes Received:
    149
  18. function

    function None functional
    Legend Veteran

    Joined:
    Mar 27, 2003
    Messages:
    5,270
    Likes Received:
    2,599
    Location:
    Wrong thread
    Yeah, I'd clinked through to the actual patent to (unsuccessfully) try and decipher it and should have linked it like you did.

    The bit that threw me (whom I kidding, it was all of it :x ) was that it was talking about being able to move between state based, title based, and triangle based coverage rates. But I couldn't work out whether you could do that within a draw call, based on dynamic variables.

    Tier 2 talks about per draw (definitely covered), per 16 x 16 or smaller tile (probably covered [1], I didn't see a specified size), per vertex + conditions (maybe covered [2] as per vertex probably means per triangle primitive in an e.g. strip, but can you do it based on changing vertex data?).

    And could "state based rate changes" [3] cover the results of various combiners? I mean, if the results of various combiners are deterministic (which I think they should be), then wouldn't that essentially constitute a number of "states"??

    Anyhoo...

    I hadn't thought of this, but I think you're absolutely on the money. MS have been working on Hololense since long before XSX, and so their interest in hardware implementations of this type are far from hypothetical. It wouldn't surprise me in the least if MS were ahead of AMD in this particular area, which might make partnering with MS wise, especially given how far behind Nvidia they have been.

    Just another thought, by MS would probably want this stuff in their (potentially) millions of XCloud devices over the next few years. Any saved performance is saved money on power consumption and cooling of enormous server clusters. Like with Hololense, it's not just about buying the silicon it's about the power it needs to run.

    This is one of the things I've been trying to work out. o_O The AMD patent states that changing shading rate per primitive is covered, and you'd definitely know where in screen space that was, and what angle the normal was to the camera, after transform.

    The question is, does the AMD patent mean you can change the shading rate based on a primitive's value, or that you can only bake it in during asset creation submission to to the draw call?

    "a triangle-based rate determination technique, in which a particular shading rate is assigned to each primitive"

    But when is it assigned, AMD? When you make the asset submit the call, or at the end of your vertex shader operation transform or later? Bah! :yelling:

    Edit: because I can't handle to even vaguely related things at once without getting confused.
     
    #3138 function, Jul 2, 2020
    Last edited: Jul 2, 2020
    tinokun, BRiT and iroboto like this.
  19. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    10,448
    Likes Received:
    10,119
    Location:
    The North
    Can it change the shading rate in the same draw as i understand is the main difference between 1 and 2
     
    Dictator and function like this.
  20. function

    function None functional
    Legend Veteran

    Joined:
    Mar 27, 2003
    Messages:
    5,270
    Likes Received:
    2,599
    Location:
    Wrong thread
    Same here, but I have no idea whether the AMD patent covers it. :no:

    Probably time for me to let smarter minds ponder this.:smile2:
     
    iroboto likes this.
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...