XBox One Backwards Compatibility and Xbox One X Enhancements for X360 and OG (XO XOX BC)

[tuna]

Each Individual X360 Game set to run on XBox One through the Backwards Compatible layer. This does not involve using the original game source code and compiling to target the Xbox One [recompilation step would be here, but it does not happen]. Instead, the binary artifacts of the game have been put through an ahead of time static translation to shift it into something suitable to run on the x64 Xbox One Backwards Compatible layer. The results of this are then repackaged as a x64 Xbox One game.

Compilation is the same as translation. The usual form of compilation is source code to machine code, but machine code to machine code is compilation as well.

 

[BRiT]

Compilation is the same as translation. The usual form of compilation is source code to machine code, but machine code to machine code is compilation as well.

I know that, but what others have been saying and hinting at is "No wonder it takes so long to get new games, because they have to recompile from source". I am making the technical distinction that "re-compile" in that regards is absolutely not happening. The MS Xbox BC has absolutely no access to the games source code and does not need it.

I'm trying to have a technical discussion that requires technical distinctions to be made but with everyone redefining specific terms to be more general then they really ought to be makes having a technical discussion impossible. Hence my frustration. They might as well be using the term "smurf" to refer to every item that is being done.

First they resmurf this, then they smurf that, then the resmurf it.

 

[Cyan]

I know that, but what others have been saying and hinting at is "No wonder it takes so long to get new games, because they have to recompile from source". I am making the technical distinction that "re-compile" in that regards is absolutely not happening. The MS Xbox BC has absolutely no access to the games source code and does not need it.

I'm trying to have a technical discussion that requires technical distinctions to be made but with everyone redefining specific terms to be more general then they really ought to be makes having a technical discussion impossible. Hence my frustration. They might as well be using the term "smurf" to refer to every item that is being done.

First they resmurf this, then they smurf that, then the resmurf it.

Certainly, I am not as technical as you, that's more than obvious, but you simply forgot the simple things. I am likely to not being reasonable if I have a technical discussion with you and many B3D fellows who know their stuff. Even so, in this discussion you remind me of one of the Network subject teachers, which is a machine and superb intelligent, and built the entire network in the school, and he was recently challenged by a classmate about something he said about WiFi.

He said he'd study the case. Next day he returned and told my classmate that he was right, so kudos to him.

But yes, it's cool to argue with me..

 

[Shifty Geezer]

Sorry but both of you are wrong, there is recompilation --whether it is from Little Endian to Big Endian or viceversa, I don't know, but there is-. If what's bolded isn't recompilation, what is it then? From DF:

That's the first time (I recall) I've seen it called PPC to x64 recompilation. It's always been referred to as 360 to XB1 repackaging. That contradicts what MS have been reiterating. From your Spencer quote, they run the XB360 as a hardware emulation. Therefore there's no need to convert the CPU code if the CPU is emulated. If they rebuild the game code from PPC to for XB1's x86 architecture, Spencer's comment would be false. I conclude Mike Rayner was incorrect.

The reason why games have to be redownloaded is because they are presented to the XB1 as an XB1 title. They need to be packaged in a format that fits the XB1 OS so that they are accessible like any other game. The alternative would be to boot into 360 mode and run the emulator as a whole system. This would allow (theoretically) full BC, put in any disc and play. But it would also need an XB360 partition on the HDD to install stuff too, and you'd lose access to the whole XB1 side of things, so no notifications or snapped apps, etc. MS want the BC to run within the XB1 experience, so need to restructure the game packages to fit the XB1 OS.

 

[Globalisateur]

Eurogamer published an article about XB1 BC, I didn't see it here:

EDIT: Sorry Cyan already quoted it here, without link though.

http://www.eurogamer.net/articles/digitalfoundry-2015-fallout-3-is-an-xbox-one-back-compat-showcase

 

[BRiT]

Le Sigh. I wish Leadbetter had simply called it what it is "machine code translation" or at the very least call it "resmurfing".

 

[Shifty Geezer]

People are getting tripped up on terms. The meat of the issue is whether the PPC code is going into the emulator and being run in real time, or whether the PPC code is being turned into x86 and x86 code is going into the emulator. Is the XB1 BC package for each title in native PPC machine language or x86, requiring preprocessing on MS's end?

That makes the difference between BC conceptually being able to run disc titles like a virtual 360 (even if not implemented as such), or whether the BC absolutely requires a preprocessing step which is one of the things limiting what titles are being made available.

 

[Cyan]

That's the first time (I recall) I've seen it called PPC to x64 recompilation. It's always been referred to as 360 to XB1 repackaging. That contradicts what MS have been reiterating. From your Spencer quote, they run the XB360 as a hardware emulation. Therefore there's no need to convert the CPU code if the CPU is emulated. If they rebuild the game code from PPC to for XB1's x86 architecture, Spencer's comment would be false. I conclude Mike Rayner was incorrect.

The reason why games have to be redownloaded is because they are presented to the XB1 as an XB1 title. They need to be packaged in a format that fits the XB1 OS so that they are accessible like any other game. The alternative would be to boot into 360 mode and run the emulator as a whole system. This would allow (theoretically) full BC, put in any disc and play. But it would also need an XB360 partition on the HDD to install stuff too, and you'd lose access to the whole XB1 side of things, so no notifications or snapped apps, etc. MS want the BC to run within the XB1 experience, so need to restructure the game packages to fit the XB1 OS.

Okay, this is what Phil Spencer said. Yet people don't know how it actually works, right? Reading your post I get I was wrong when I said it's not a hardware compiler, but the question stands, what's being compiled and whatnot. Maybe nothing is compiled and you and BriT are right, but I highly doubt that's the case.

http://www.thetechgame.com/News/sid...w-xbox-one-backwards-compatibility-works.html

"Millions of people made investments in 360 content," he said. "We thought the right thing to do was to make that content go forward, but we didn't know [how difficult it would be]."

"[Emulation] is hard," admitted Spencer, explaining that the company was dealing with having to harmonise PowerPC architecture with x86.

"The approach that we've taken is to actually emulate the full Xbox 360 hardware layer. So the [operating system] for the 360 is actually running when you run the game," Spencer explained.

"If you watch the game's boot you'll see the Xbox 360 boot animation come up. From a performance standpoint it allows [emulation] to work. We're able to get frame by frame performance equivalents."

"[Xbox Live] thinks you're on a 360, so people have been asking 'hey, why are you playing Mass Effect on the 360?,' I was actually playing on the Xbox One."

Spencer continued to explain that, since the Xbox One thinks it's playing a normal game, features such as streaming and screenshots are supported.

"The 360 games think they're running on the 360 OS, which they are. And the 360 OS thinks its running on the hardware, which it's not, it's running on an emulated VM. On the other side, the Xbox One thinks it's a game. That's why things like streaming, game DVR, and screenshots all work, because it thinks there's just one big game called 360."

Delving deeper, Spencer explained exactly how the emulator packages the Xbox 360 games, and how it compares to Xbox 360's emulation of original Xbox games.

"You download a kind of manifest of wrapper for the 360 game, so we can say 'hey, this is actually Banjo, or this is Mass Effect. The emulator runs exactly the same for all the games.

"I was around when we did the original Xbox [backwards compatibility] for Xbox 360 where we had a shim for every game and it just didn't scale very well. This is actually the same emulator running for all of the games. Different games do different things, as we're rolling them out we'll say 'oh maybe we have to tweak the emulator.' But in the end, the emulator is emulating the 360, so it's for everybody."

Asked about whether Microsoft would require permission from game publishers to adjust game code, Spencer clarified it would not be interfering with code.

"The bits are not touched," he said. "There's some caveats, and as always I like to be as transparent as I can be on this: Kinect games won't work from the 360, because translating between the Kinect sensors is almost impossible."

Finally, the subject of multi-disc games was also addressed. According to Spencer, it's an issue engineers are looking into.

"We're still working on multi-disc," he said. "Lost Odyssey and Blue Dragon are some of my favourites from the 360. There's actually work in packing a multi-disc into single that requires us to go back and look at the original package on the multiple discs and reconfigure that."

_________________________________________________________________________________________________________________________________

This is what Rayner said (from Globalisateur link):

"It is essentially the exact same code," Rayner replied. "The Xbox team converts the 360 game and 360 flash PPC executables into native x64 executables, packages those up with the 360 game assets, 360 flash and emulator as a regular Xbox One game, and publishes it."

 

[Cyan]

Le Sigh. I wish Leadbetter had simply called it what it is "machine code translation" or at the very least call it "resmurfing".

Is this what you are talking about..binary translation? I guess @tuna was correct then.

https://en.wikipedia.org/wiki/Binary_translation

In computing, binary translation (or (binary) recompilation)

A translator using static binary translation aims to convert all of the code of an executable file into code that runs on the target architecture without having to run the code first, as is done in dynamic binary translation. This is very difficult to do correctly, since not all the code can be discovered by the translator. For example, some parts of the executable may be reachable only through indirect branches, whose value is known only at run-time.

Static binary translation[edit]
A translator using static binary translation aims to convert all of the code of an executable file into code that runs on the target architecture without having to run the code first, as is done in dynamic binary translation. This is very difficult to do correctly, since not all the code can be discovered by the translator. For example, some parts of the executable may be reachable only through indirect branches, whose value is known only at run-time.

One such static binary translator uses universal superoptimizer peephole technology (developed by Sorav Bansal, and Alex Aiken from Stanford University) to perform efficient translation between possibly many source and target pairs, with considerably low development costs and high performance of the target binary. In experiments of PowerPC-to-x86 translations, some binaries even outperformed native versions, but on average they ran at two-thirds of native speed.

 

[Persistantthug]

I'm trying to understand how they did it.

Before I was under the impression that it couldn't be done because the Power PC Cpu was too different from an X86 cpu, and since alot of games used the Cpu, more or less, for some graphics processing, that it would be quite a while until XBOX 360 games could be emulated.

I know that it will be many times harder for any CELL PROCESSOR emulation, but it should still be difficult to emulate a Power PC cpu......at least that was my understanding.


So, How'd they do it?

Was it Sorcery?

Voodoo?

 

[tuna]

If we leave semantics aside, are there any reports of the performance of the final BC solution (aside from DF's Fallout 3 report)?

 

[Persistantthug]

Now that I've typed this, Perhaps I have my own hypothesis......

Is Microsoft actually reprogramming and/or porting each and every XBOX 360 game with their API (Direct X 12?) ?

Is that what is going on, Beyond3d guys?

 

[BRiT]

Microsoft are not changing any of the Game code at all. They are not porting the game code. To do so would require access to the game source code. They are not using the source code. They are using the binary artifacts of the game.

What they have done is port the entire X360 OS and libraries that the games are using, onto the Xbox One as a "game".

The binary artifacts of the game is processed through a static translation and then repackaged along with the "Xbox 360 BC Layer" to be treated as an Xbox One Game.

 

[Persistantthug]

Microsoft are not changing any of the Game code at all. They are not porting the game code. To do so would require access to the game source code. They are not using the source code. They are using the binary artifacts of the game.

What they have done is port the entire X360 OS and libraries that the games are using, onto the Xbox One as a "game".

The binary artifacts of the game is processed through a static translation and then repackaged along with the "Xbox 360 BC Layer" to be treated as an Xbox One Game.

Ok......
I know what binary is.....
I know what an "artifact" is.


But I'm gonna need a little help......
What is a Binary Artifact?

lol

Seriously though.....
what is it?

 

[BRiT]

Binary Artifact is the digital artifacts of a game. Perhaps I should have used "digital artifacts", but it's akin to "binary executable" to distinguish it from "source code".

In the typical software development world, you take the source code and compile them to generate artifacts, typically a binary executable or binary file. I wont get into systems that use compilers to generate intermediate code that is interpreted at runtimes like JVM or MS CLR nor will I get into systems that use preprocessors like M4 or T4 to expand macros.

 

[BadTB25]

Looking forward to playing Halo Reach when it is available for BC. I might even play through ODST again, but it is my least favorite Halo.

 

[SedentaryJourney]

Binary Artifact is the digital artifacts of a game. Perhaps I should have used "digital artifacts", but it's akin to "binary executable" to distinguish it from "source code".

In the typical software development world, you take the source code and compile them to generate artifacts, typically a binary executable or binary file. I wont get into systems that use compilers to generate intermediate code that is interpreted at runtimes like JVM or MS CLR nor will I get into systems that use preprocessors like M4 or T4 to expand macros.

So the machine code or binary object code of the game is translated ahead of time, and it runs on top of a virtual Xbox360?

 

[Cyan]

From now on, BC games are going to be published immediately when they are ready rather than as a part of a monthly list.

http://www.techradar.com/news/gamin...to-xbox-one-s-backwards-compatibility-1313588

next game, seems to be Call of Duty Black Ops http://segmentnext.com/2016/01/23/call-of-duty-black-ops-is-backwards-compatible-on-xbox-one/

I want Bioshock Infinite (unlocked framerate, nice), Skyrim and Call of Juarez (unlocked framerate too, great, maybe it will run faster on Xbox One)

 

[xexuxjy]

I'd still like to see some of the older jrpgs like Lost Odyssey and Blue Dragon, which as they're MS published I thought would be easier to get going (multi-disc issues for LO aside). Also be good to see more indy games (Deathspank!) coming over, though the last update re-enabled Age of Booty and Space Giraffe for me so that was good.

 

[milk]

Of course we all know the original 360 binaries are being resmurfed into native x86 code, but its important to understand what smurfing type they are using. Papa smurf, for one, wears red clothing, or Smurfette, she is female. That's the final piece of the puzzle we are trying to uncover here.