Why has the xbox not been emulated up to now?
- Thread starter Commenter
- Start date
D
Deleted member 11852
Guest
Personally I would be reluctant to put time and effort into such a project unless Microsoft and Nvidia gave me some kind of re-assurance not to get legal on my arse. I can't see any reason for Microsoft or Nvidia to do this.
I suspect that even Rys would ban me if I were to give an appropriate image response in answer to your question, so let me just say that nVidia is the devil and whatever happened in Xbox, Itagaki afskajfd;aijds;lfahsdlfhaldsflakf;lakjsfdlkajsdf;lkjasdffadsfasdfasdf
The GPU was using same "building blocks" as GF3, so surely there's enough documentation
"Xbox is just like a PC, it's easy to emulate!"
Yes, we've all heard this silly and pointless argument a million times and it usually ends in the same, and rather ignorant conclusion (or should I say assumption) that just because the Xbox is PC similar, it's hardware should be relatively easy to emulate. That's a very wrong frame of mind. How hard can it be? Very. Xbox's hardware is very complex and still poorly documented to this day. This requires some explanation.
1. Is a PC easy to emulate? Well, I wouldn't say so myself. Take a look at the source code from bochs. A lot of source code/work isn't it?
2. Emulating an x86 CPU is a lot harder than it sounds. I don't know where this mindless assumption comes from. Yes, there's loads of documentation on how the x86 processor works, but that doesn't exactly make it easy. First of all, the x86 instruction set is M-A-S-S-I-V-E! There can be at least 20 different versions of one instruction (i.e. There are many different versions of the MOV instruction, as well as INC, DEC, ADD, SUB, SHR, SHL, AND, OR, XOR etc.) and it takes time to implement them all. Of course, that's not exactly difficult. The real problem is that any modern x86 processor including the Pentium III can execute multiple instructions at once. So it's not like emulating a Z80 doing one instruction at a time. The actual algorithm and how x86 does this is undocumented and still unknown. In short, the Xbox's CPU can be emulated, but not accurately.
3. Emulating any hardware by NVIDIA is not a walk in the park! The Xbox's GPU, the NV2A is often assumed just a GeForce 3. It's not! It's similar but not identical. It has some GeForce 4 capabilities too, so it's more of a cross between an NV20 and NV25. This is by no means easy to emulate either. NVIDIA's GPUs have very large register sets and afaik not even half of them have been discovered, and a large portion of known registers have unknown purposes. There is little to no documentation on how NVIDIA GPUs work. The best thing to do is to look at similar GPUs such as RIVA, TNT, and older GeForce cards. Some registers are similar, but not identical. The best place to look for information is in open source drivers available on the net. Adding to the dificulty is that no one has ever discovered how pixel shaders work on NV2x cards, vertex shaders yes though. The Xbox GPU also has exclusive registers that are not found in other GeForce cards. Information on the NV2A's GPU registers are just now beginning to be discovered a few months ago. And yet, there's still a long way to go. The GeForce 3 series is the most mysterious of all NVIDIA GPUs (G7x and G8x aside) and the NV2A is alot worse. "But can't you just directly execute the NV2A instructions on another NVIDIA card?". No, I get alot of questions concerning this, and it is impossible. It's MMIO addresses are different and the exclusive registers must be emulated. Plus, in windows, we don't have ring 0 access anyway, so you all can scratch that idea now. Then comes the NForce 2 chipset. This is where it get easier. The NVIDIA MCPX is the control center for things such as audio, USB for input, Network adapters, PCI, AGP, etc. These things are not really that difficult to emulate IMO except for the audio.
4. The Audio system is rather complex. Xbox's audio consists of at least 4 DSPs, and audio codec (AC '97) and an NVIDIA SoundStorm APU. The DSPs shouldn't be a problem (just figuring out what they all are is) nor should the AC '97 but the NVIDIA SoundStorm APU is the really difficult part. So far I haven't found any information on this thing, but right now, it's relevance is low.
5. The Xbox BIOS isn't fully understood. The basic execution process of the BIOS is understood, but details on the process are at a loss. What we do know gives us hints, but before the BIOS can be emulated, we'll need a better understanding of the Xbox hardware layout because the BIOS does some unknown hardware initialization at boot time and writes to the hardware directly without using any XDK stuff. It will take some time, and effort, but I'll eventually get it working.
6. Video Encoder "Hell". Instead of using a RAMDAC for video output, the Xbox uses a Video Encoder. What makes this suck a pain? Microsoft sought the need to change the video encoder every other Xbox version (there are seven in all, 1.0 - 1.6). Why, I dunno, it's a Microsoft thing, they always tend to try to "fix" things that aren't broken >.> AFAIK, there are at least 3 different Video Encoders used: Conexant CX25871, Focus FS454, and Xcalibur. For more information in Xbox video encoders, click here. Emulating all three video encoders is only less than half the battle, the real problem is that BIOSes can be specifically tied to a specific encoder depending on it's version (don't quote me on this though). Like PS2, every Xbox model revision has a updated BIOS and has different expectations. This is a potential problem, but not exactly major.
Basically, I'm trying to get this "Xbox should be easy to emulate because it's just like a PC" crap out of your heads. I'm sure that most of you will disagree with me on this, but for these reasons and more, on a low level, Xbox is harder to emulate than PS2.
^ The exception to that above is that the NV2A is much better documented now, but not fully. There's an open source library for the OpenXDK called pbKit. It interfaces directly with the hardware to fully expose it's potential. This is what Microsoft should have done all along (or at least wrote a low-level OpenGL implementation).
From NGEMU.
Yes, we've all heard this silly and pointless argument a million times and it usually ends in the same, and rather ignorant conclusion (or should I say assumption) that just because the Xbox is PC similar, it's hardware should be relatively easy to emulate. That's a very wrong frame of mind. How hard can it be? Very. Xbox's hardware is very complex and still poorly documented to this day. This requires some explanation.
1. Is a PC easy to emulate? Well, I wouldn't say so myself. Take a look at the source code from bochs. A lot of source code/work isn't it?
2. Emulating an x86 CPU is a lot harder than it sounds. I don't know where this mindless assumption comes from. Yes, there's loads of documentation on how the x86 processor works, but that doesn't exactly make it easy. First of all, the x86 instruction set is M-A-S-S-I-V-E! There can be at least 20 different versions of one instruction (i.e. There are many different versions of the MOV instruction, as well as INC, DEC, ADD, SUB, SHR, SHL, AND, OR, XOR etc.) and it takes time to implement them all. Of course, that's not exactly difficult. The real problem is that any modern x86 processor including the Pentium III can execute multiple instructions at once. So it's not like emulating a Z80 doing one instruction at a time. The actual algorithm and how x86 does this is undocumented and still unknown. In short, the Xbox's CPU can be emulated, but not accurately.
3. Emulating any hardware by NVIDIA is not a walk in the park! The Xbox's GPU, the NV2A is often assumed just a GeForce 3. It's not! It's similar but not identical. It has some GeForce 4 capabilities too, so it's more of a cross between an NV20 and NV25. This is by no means easy to emulate either. NVIDIA's GPUs have very large register sets and afaik not even half of them have been discovered, and a large portion of known registers have unknown purposes. There is little to no documentation on how NVIDIA GPUs work. The best thing to do is to look at similar GPUs such as RIVA, TNT, and older GeForce cards. Some registers are similar, but not identical. The best place to look for information is in open source drivers available on the net. Adding to the dificulty is that no one has ever discovered how pixel shaders work on NV2x cards, vertex shaders yes though. The Xbox GPU also has exclusive registers that are not found in other GeForce cards. Information on the NV2A's GPU registers are just now beginning to be discovered a few months ago. And yet, there's still a long way to go. The GeForce 3 series is the most mysterious of all NVIDIA GPUs (G7x and G8x aside) and the NV2A is alot worse. "But can't you just directly execute the NV2A instructions on another NVIDIA card?". No, I get alot of questions concerning this, and it is impossible. It's MMIO addresses are different and the exclusive registers must be emulated. Plus, in windows, we don't have ring 0 access anyway, so you all can scratch that idea now. Then comes the NForce 2 chipset. This is where it get easier. The NVIDIA MCPX is the control center for things such as audio, USB for input, Network adapters, PCI, AGP, etc. These things are not really that difficult to emulate IMO except for the audio.
4. The Audio system is rather complex. Xbox's audio consists of at least 4 DSPs, and audio codec (AC '97) and an NVIDIA SoundStorm APU. The DSPs shouldn't be a problem (just figuring out what they all are is) nor should the AC '97 but the NVIDIA SoundStorm APU is the really difficult part. So far I haven't found any information on this thing, but right now, it's relevance is low.
5. The Xbox BIOS isn't fully understood. The basic execution process of the BIOS is understood, but details on the process are at a loss. What we do know gives us hints, but before the BIOS can be emulated, we'll need a better understanding of the Xbox hardware layout because the BIOS does some unknown hardware initialization at boot time and writes to the hardware directly without using any XDK stuff. It will take some time, and effort, but I'll eventually get it working.
6. Video Encoder "Hell". Instead of using a RAMDAC for video output, the Xbox uses a Video Encoder. What makes this suck a pain? Microsoft sought the need to change the video encoder every other Xbox version (there are seven in all, 1.0 - 1.6). Why, I dunno, it's a Microsoft thing, they always tend to try to "fix" things that aren't broken >.> AFAIK, there are at least 3 different Video Encoders used: Conexant CX25871, Focus FS454, and Xcalibur. For more information in Xbox video encoders, click here. Emulating all three video encoders is only less than half the battle, the real problem is that BIOSes can be specifically tied to a specific encoder depending on it's version (don't quote me on this though). Like PS2, every Xbox model revision has a updated BIOS and has different expectations. This is a potential problem, but not exactly major.
Basically, I'm trying to get this "Xbox should be easy to emulate because it's just like a PC" crap out of your heads. I'm sure that most of you will disagree with me on this, but for these reasons and more, on a low level, Xbox is harder to emulate than PS2.
^ The exception to that above is that the NV2A is much better documented now, but not fully. There's an open source library for the OpenXDK called pbKit. It interfaces directly with the hardware to fully expose it's potential. This is what Microsoft should have done all along (or at least wrote a low-level OpenGL implementation).
From NGEMU.
...on an x86 PC. I think you missed that part.
On a PC, it's zero effort.
Using an x86 CPU, you don't need to emulate x86.
This is the difficult part, but where the API is DX based (and the name XBox comes from DirectX Box), it should be doable. How easy and whether it's worth it are the defining questions to my mind, but it seems pretty implausible that a modern DX capable part can't emulate NV2A's instructions/shaders
Hmm, you can add me to that list.
There's nothing in common between PS2 hardware and a modern Windows PC. There's a fair bit in common between the x86 CPU, nVidia GPU, DirectX based Xbox and a modern Windows PC. It's probably fair to say people underestimate the difficulty, but XB is harder to emulate than PS2?[/quote]
Last edited:
pMax
Regular
...BS apart?
PS is using linked libs, xb is embedding them. Now, a thing is to rewrite&emulate the bindings and make some exception/patches, a thing is to emulate a full machine.
How easy do you think it is emulating DX calls vs. getting its functions INSIDE a binary and then do the same? (note: it is a rhetorical question)
PS is using linked libs, xb is embedding them. Now, a thing is to rewrite&emulate the bindings and make some exception/patches, a thing is to emulate a full machine.
How easy do you think it is emulating DX calls vs. getting its functions INSIDE a binary and then do the same? (note: it is a rhetorical question)
I believe that quote was declared out of date by the guy who originally wrote it.
More is known about the gpu and the most advanced emulator (xqemu) has taken the low level emulation route, sidestepping the issue with handling the many xdk versions.
There's two parts to this argument really..
First one is about the x86 CPU. You could emulate x86 on x86 using virtualization. But this has a lot of its own challenges and limitations that a traditional recompiler doesn't. It'd probably ultimately serve as more of a performance boost than anything.
The other is about how you handle everything else. People say that it uses Windows, DirectX, a "PC-like" GPU and so on so it should be easy to emulate "directly." Doing so involves HLE (high level emulation), or performing emulation of library, OS, BIOS etc functions instead of their low level instructions. This has been attempted a lot in XBox emulation, starting with some real results over over 10 years. It has been found that there are a ton of obstacles with this approach. The SDK comes on the game disk itself, was huge, and there were many revisions that subtly change things. This makes tracking and emulating all of it a very large and insurmountable problem. And if the games sidestep the SDK and access things directly then this approach will fail. For this reason pretty much everyone I've seen involved in XBox emulation has eventually agreed that full HLE is not the best approach.
If you're not using HLE that means you're emulating all the hardware peripherals at the I/O level and then it doesn't matter if the thing is running DirectX or Windows or if it looks similar to some PC nVidia card from a long time ago. That might give some better understanding as to what it's doing and there could be some shared documentation effort that helps. It could also make the actual hardware functionality easier to fully map to PC hardware. But in terms of effort level it doesn't really make emulating those peripherals easier than emulating another console's, and in XBox's case the GPU is fairly complex compared to most other things that have been emulated. And the GPU is only one of many peripherals, including some that have just not been reverse engineered that well.
In a way, XBox's PC heritage makes things harder for low level emulation because it has all this cruft that a hardware platform designed specifically and entirely for the console wouldn't.
First one is about the x86 CPU. You could emulate x86 on x86 using virtualization. But this has a lot of its own challenges and limitations that a traditional recompiler doesn't. It'd probably ultimately serve as more of a performance boost than anything.
The other is about how you handle everything else. People say that it uses Windows, DirectX, a "PC-like" GPU and so on so it should be easy to emulate "directly." Doing so involves HLE (high level emulation), or performing emulation of library, OS, BIOS etc functions instead of their low level instructions. This has been attempted a lot in XBox emulation, starting with some real results over over 10 years. It has been found that there are a ton of obstacles with this approach. The SDK comes on the game disk itself, was huge, and there were many revisions that subtly change things. This makes tracking and emulating all of it a very large and insurmountable problem. And if the games sidestep the SDK and access things directly then this approach will fail. For this reason pretty much everyone I've seen involved in XBox emulation has eventually agreed that full HLE is not the best approach.
If you're not using HLE that means you're emulating all the hardware peripherals at the I/O level and then it doesn't matter if the thing is running DirectX or Windows or if it looks similar to some PC nVidia card from a long time ago. That might give some better understanding as to what it's doing and there could be some shared documentation effort that helps. It could also make the actual hardware functionality easier to fully map to PC hardware. But in terms of effort level it doesn't really make emulating those peripherals easier than emulating another console's, and in XBox's case the GPU is fairly complex compared to most other things that have been emulated. And the GPU is only one of many peripherals, including some that have just not been reverse engineered that well.
In a way, XBox's PC heritage makes things harder for low level emulation because it has all this cruft that a hardware platform designed specifically and entirely for the console wouldn't.
The reason would be that they don't actually have a legal standing against emulators. I'm not aware of anyone successfully winning a lawsuit against an emulator. Sony tried it in the 90s and they lost; while the litigation was disastrous for their opponents it established case law and Sony (or AFAIK anyone else) hasn't tried that since.
These days when you see an emulator get taken down from somewhere it's because they bundled ROMs or linked to ROM websites, or because they used trademarked images somewhere.
D
Deleted member 11852
Guest
I think the guy is referring to cycle-perfect emulation of what seems to be a variation of a Coppermine-128 chip. This can mean software emulation of the entire architecture and cache. Remember, the goal is not compatibility (relatively easy) but emulation so exact that the OS and software can't tell the difference and doesn't break or introduces frame pacing issues or other graphical artefacts.
I don't think anyone emulating a console anywhere remotely on the level of XBox has a goal like that. Nothing so complex has such an accurate emulator out.
But, going with virtualization or any other kind of "direct" execution is on the extreme opposite end of timing accuracy. Not only would it tend to be very far off but you'd get results that are completely different depending on what machine it's executed on. In particular, if the emulation is too slow it can break (as in, prevent from working) even a sanely coded game, and only on some machines. This could be difficult to impossible to reproduce on another computer. Now imagine that you do run it on a computer that's nominally fast enough, but there's some sudden spike in CPU pressure from other things running on the machine and you get the same thing, crashing game.
These are just some of the problems with virtualizing console games.
This is actually not true.
There's only one case I've ever heard of, and that's UltraHLE. Nintendo sent a C&D letter and the team acquiesced.
There have been several emulators of Nintendo platforms (NES, SNES, Gameboy) before UltraHLE and none of them received such letters. And there were several (in many ways superior) N64 emulators out not very long after UltraHLE and they didn't receive letters either. Maybe Nintendo realized that they didn't actually have a legal standing afterall. Probably the only reason they went after UltraHLE in the first place was because it was released while N64 games were still selling in pretty big numbers; AFAIK it was just a few months after Ocarina of Time came out. You could hardly say the same thing for an XBox emulator today.
There was a claim at one point that a GBA emulator for Palm platforms like Tapwave Zodiac got a letter from Nintendo, but I think it was just a stalling tactic on the developer's part because they had promised something they were nowhere close to able to deliver. And they did eventually release something anyway, and it was nowhere close to what they said it'd be. I also did a GBA emulator for a mobile platform around the same time and didn't get a letter from Nintendo.
D
Deleted member 11852
Guest
Stating the obvious but there is the PS2, where early emulators also ran slow because of the need of exact timings. You're not emulating a single CPU, you're emulating a full machine state.
Full machine state emulation is something we do a lot where I work. We're not interested in emulating consoles of course, but we do a lot of machine of security bespoke security hardware which presents very similar problems
Its interesting that the 1st person (Ben Vanik) to make a Microsoft console emulator is located in Japan. Many of its other contributors are also not located in english speaking countries either. Eg John Godgames
Ben said one of his motivations was him was the inability to find 360 games in Japan, however, for the amount of time he's put into the emulator he could easily have imported the titles not available on the PS3.
Ben said one of his motivations was him was the inability to find 360 games in Japan, however, for the amount of time he's put into the emulator he could easily have imported the titles not available on the PS3.
Last edited:
D
Deleted member 13524
Guest
By now I would say just the lack of interest and fear of nVidia's legalese trolls.
The XBone could probably emulate 5 xboxes at the same time with little effort.
The XBone could probably emulate 5 xboxes at the same time with little effort.
