Xbox One (Durango) Technical hardware investigation

[tunafish]

JPEG is actually a horrible format to keep your textures in, for one you will blow out your cache and memory due to the fact that it only decompresses them and that video card's dont have native decompression in the TEX units for JPEG. For second it was pretty explicitly mentioned that it was there to help with Kinect processing, and that was it really.

JPEG is not a texture compression format -- you can't do random access in it, so you *cannot* do native decompression in the cards like you do with texture compression algorithms. However, JPEG is somewhere near 10x more space-efficient for similar quality than the best of the texture compression algorithms.

The way you use JPEG is with virtual texturing -- basically, you keep most of your textures stored as JPEGs, and decompress blocks of those into the in-use texture atlas when they are needed.

 

[Betanumerical]

JPEG is not a texture compression format -- you can't do random access in it, so you *cannot* do native decompression in the cards like you do with texture compression algorithms. However, JPEG is somewhere near 10x more space-efficient for similar quality than the best of the texture compression algorithms.

The way you use JPEG is with virtual texturing -- basically, you keep most of your textures stored as JPEGs, and decompress blocks of those into the in-use texture atlas when they are needed.

True, but the iD way seems to be using JPEG 2000, which is the XBONE doesn't support, it only supports JPEG 1994. I'm not sure how much use JPEG 1994 is for games.

I have yet to come across anything (aside from the JPEG 2000 recommendation for Virtual Texturing) that suggests using JPEG, everything seems to suggest using BCn.

 

[adev]

True, but the iD way seems to be using JPEG 2000, which is the XBONE doesn't support, it only supports JPEG 1994. I'm not sure how much use JPEG 1994 is for games.

I have yet to come across anything (aside from the JPEG 2000 recommendation for Virtual Texturing) that suggests using JPEG, everything seems to suggest using BCn.

I wonder why the Xbox One would have a co-processor capable of decoding frames of JPEG data into tiled YUV formats such as dxgi_format_g8r8_g8b8_unorm.

Would GCN have instructions to sample such data?

 

[Brad Grenz]

Doesn't Kinect use MJPEG for the color camera? Would not the JPEG decoder on the Move Engine just be a convenient, low overhead way to decode that in realtime for image analysis and other processing?

 

[adev]

Doesn't Kinect use MJPEG for the color camera? Would not the JPEG decoder on the Move Engine just be a convenient, low overhead way to decode that in realtime for image analysis and other processing?

According to this it's YUY2 which presumably means there's no need to decode it using the move engine either: http://www.vgleaks.com/durango-next-generation-kinect-sensor/

I doubt it's a coincidence that the only move engine with JPEG decode capabilities is the one which is also shared with the system. Given how those engines are connected via busses, I can't imagine many games will lean too heavily on that capability.

It'd be interesting to know how the time slicing affects the co-processors, if it's just the CPU and GPU which are time sliced then presumably the OS could kick off Async move jobs which overrun the time slice.

EDIT: It's always possible that Kinect is capturing YUY2 frames, encoding it on the camera, sending it to the device then decoding it at the other end.

 

[dumbo11]

EDIT: It's always possible that Kinect is capturing YUY2 frames, encoding it on the camera, sending it to the device then decoding it at the other end.

Yep, the diagrams from their presentation actually label the part as 'MJPEG decode'.

 

[Grall]

EDIT: It's always possible that Kinect is capturing YUY2 frames, encoding it on the camera, sending it to the device then decoding it at the other end.

That'd be weird - reports say kinect2 is USB3 capable, so there should be no need for any encoding. There's plenty bandwidth to just dump over raw frames across the bus. An encode-decode process just increases latency, and in a system like kinect you really don't want that.

 

[adev]

I agree.

Tin foil hats on.

JPEG decode for run length Golomb Rice?

For reference: http://research.microsoft.com/apps/mobile/publication.aspx?id=102069

 

[Shifty Geezer]

An encode-decode process just increases latency...

And compression artefacts that negatively impact image recognition and processes.

 

[Gubbi]

It doesn't use compression, AFAICT. It uses YUY2 which is just 4:2:2 chroma subsampling with a specific sequence of luminance and chrominance values (Y, Cb, Y, Cr or something like that)

Cheers

 

[adev]

It doesn't use compression, AFAICT. It uses YUY2 which is just 4:2:2 chroma subsampling with a specific sequence of luminance and chrominance values (Y, Cb, Y, Cr or something like that)

Cheers

I agree, it was the only source of JPEG images I could think outside of plain system UI images.

The RemoteFX codec stuff is interesting and the Xbox One looks an awful lot like a box designed to handle that. Pure speculation, not sure where crazy ideas fit but it would fee related to the JPEG move engine if it can handle the lossless part of the 1994 JPEG spec.

 

[Shifty Geezer]

It doesn't use compression, AFAICT. It uses YUY2 which is just 4:2:2 chroma subsampling

Technically, that's a form of lossy compression.

 

[Gubbi]

Technically, that's a form of lossy compression.

Not when it is the format that is read out of the sensor.

Cheers

 

[dobwal]

True, but the iD way seems to be using JPEG 2000, which is the XBONE doesn't support, it only supports JPEG 1994. I'm not sure how much use JPEG 1994 is for games.

I have yet to come across anything (aside from the JPEG 2000 recommendation for Virtual Texturing) that suggests using JPEG, everything seems to suggest using BCn.

I read that ID used JPEG XR for RAGE. It may have been a question of bandwidth consumption versus decompressing rates. JPEG 2000 and XR provide better quality at higher compression ratios but at the cost of being more computationally expensive.

MS may have picked JPEG standard because it provides sufficient quality at the decompression rate needed by XB1.

 

[dumbo11]

AFAIK the 1994 spec does include lossless jpegs, but it doesn't include things like alpha channels.

The description of the DME also refers to a single specific texture format, which I assume wasn't chosen at random...

http://www.vgleaks.com/world-exclusive-durangos-move-engines/

In the case of 4:2:2 subsampling, the luma and chroma channels are interleaved. The GPU supports special texture formats (DXGI_FORMAT_G8R8_G8B8_UNORM) and tiling modes to allow all three channels to be fetched using a single instruction, even though they are of different resolutions.

---

DXGI_FORMAT_G8R8_G8B8_UNORM
A four-component, 32-bit unsigned-normalized-integer format. This packed RGB format is analogous to the YUY2 format. Each 32-bit block describes a pair of pixels: (R8, G8, B8) and (R8, G8, B8) where the R8/B8 values are repeated, and the G8 values are unique to each pixel.

 

[scently]

AFAIK the 1994 spec does include lossless jpegs, but it doesn't include things like alpha channels.

The description of the DME also refers to a single specific texture format, which I assume wasn't chosen at random...

http://www.vgleaks.com/world-exclusive-durangos-move-engines/

It does seem that this was design with not just Kinect in mind then. It looks like they added special texture format and tiling instruction. Also note that the move engine that is shared between the game and the system isn't the one with the jpeg decode.

 

[Gipsel]

It does seem that this was design with not just Kinect in mind then. It looks like they added special texture format and tiling instruction. Also note that the move engine that is shared between the game and the system isn't the one with the jpeg decode.

As far as I can see, these formats are supported by all GCN GPUs, it's not exclusive to XB1 (and not even GCN 1.1).

 

[scently]

As far as I can see, these formats are supported by all GCN GPUs, it's not exclusive to XB1 (and not even GCN 1.1).

Oh ok. Good to know.

 

[Cyan]

I wonder what does this developer mean by "the memory write" comment.

http://www.edge-online.com/news/the...mer-feedback-made-xbox-one-a-contender-again/

“[PS4’s] DDR5 is basically 50 per cent more powerful than DDR3,” says Gaijin Entertainment CEO Anton Yudintsev. “But the memory write is bigger on Xbox One. So it depends on what you’re doing. PS4 is more powerful, but you can’t just write to the memory, you need to read sometimes.”

Xbox One now has more CPU power than PS4, while PS4 retains the advantage in GPU speed. “They maybe have a little more GPU,” says Lobb. “We have eSRAM [embedded memory] and crazy bandwidth to that eSRAM. Which is going to be better in the long run from a developer [perspective]? We’re going to see as the games go head to head. A lot of it will come down to – as always – which exclusive teams push a piece of hardware best.”

Thanks NospheratuX for the news.

 

[onQ]

I wonder what does this developer mean by "the memory write" comment.

http://www.edge-online.com/news/the...mer-feedback-made-xbox-one-a-contender-again/

Thanks NospheratuX for the news.

Also what does he mean by

“[PS4’s] DDR5 is basically 50 per cent more powerful than DDR3,”

?

PS4's 176GB/s GDDR5 is around 160% faster than Xbox One's 68GB/s DDR3.