Apple going with IMG for years to come ?

[Arun]

It is simply gradually becoming less important in the greater scheme of things as overall interest shifts to other areas.

Absolutely agreed, but AppleInsider goes quite a bit beyond that, which I find rather absurd.

How important OpenCL on mobile platforms becomes will depend on a lot of things, but I think it can be fairly argued that it is likely to be more important there than on Wintel PCs equipped with increasingly capable floating point hardware as standard.

Have you look at ARM's Neon? On the A8 it's 64-bit, but on the A9 it's 128-bit FMAC single-cycle. It's not as flexible in terms of issuing instructions as the x86 CPUs, but in terms of raw GFlops throughput per MHz it's the same as Barcelona or Nehalem

For just about every floating point code I come across, the limitation already lies in the memory subsystem rather than the FP hardware. And Intel is set to provide additional capabilities in their next generation CPUs.

If what you said was 100% true in all cases, HPC GPGPU would never deliver speed-ups significantly above memory bandwidth differences, which it definitely sometimes does. But yeah, I agree it's a substantial factor.

and having underutilized transistors is not only a cost in terms of money, but the cost in power draw is a very serious issue.

Power islands are pretty damn omnipresent nowadays, I'd argue that really isn't the biggest problem... What is much more problematic is having unused transistors *inside a given subsystem*, and presumably OpenCL could minimize that a bit by not forcing everything to be so damn flexible (rare workloads could just be offloaded to the GPU) but I don't think it should be overestimated. However it is interesting to ponder the possibility that Apple would do video encoding on the SGX 520 for their next-gen 65nm SoC, not on the ARM11.

If having OpenCL has the potential to save transistors, or even entire chips elsewhere in the system, this is quite significant. Is this the case though? I honestly don't know.

I don't see any killer app for it right now, but I'm sure eventually somebody will find something more useful to do with it than the proposed "analyze image to tell what monument I'm looking at because I'm too lazy to look at the goddamn plate in front of it or ask the locals" usage!

 

[Entropy]

Have you look at ARM's Neon? On the A8 it's 64-bit, but on the A9 it's 128-bit FMAC single-cycle. It's not as flexible in terms of issuing instructions as the x86 CPUs, but in terms of raw GFlops throughput per MHz it's the same as Barcelona or Nehalem

Yes. I can see a lot of places where the A9 in sympathetic surroundings could be a very viable alternative to x86, when binary x86 compatibility isn't a major issue.

Off topic:

If what you said was 100% true in all cases, HPC GPGPU would never deliver speed-ups significantly above memory bandwidth differences, which it definitely sometimes does. But yeah, I agree it's a substantial factor.

Even though computational science is my branch, you'll note my almost complete absence from the GPGPU forum. It just doesn't do anything for the problems at hand. This is an area where I have reason to tread lightly, but as far as I can see high performance computing have been addressing a continously shrinking set of problems with time. It is easy to understand why, since we have moved from faster FPUs and memory, to vector, to parallell, to vector-parallell, to massively parallell to vector-massively... The number of problems that effectively map to these increasingly demanding structures is naturally shrinking.
There are important problems that DO map well enough, and of course (and in my opinion this is what drives the community these days), HPC architectures and programming is an interesting technological field in its own right. What I'm typically seeing now though is HPC folks looking for problems that map to their systems, rather than the other way around.
But most scientific/technical problems do not benefit greatly from the Big Iron, and in many cases the underlying approximations are coarse enough, or the computational load light enough, that a PC can deal with the problems so well that going further just isn't justified over the utility of doing your work on a laptop.
And I say that as part of a community that burn supercomputing cycles like there is no tomorrow.

I don't see any killer app for it right now, but I'm sure eventually somebody will find something more useful to do with it than the proposed "analyze image to tell what monument I'm looking at because I'm too lazy to look at the goddamn plate in front of it or ask the locals" usage!

Indeed. In mobile space at least efficiency is still a big issue, so even modest overall efficiency gains has some value there. Whether sufficient to provide tangible benefits to the end-user remains to be seen. It seems reasonable to assume that Apple has some particular applications in mind though, and once those are revealed along with the underlying platform to support them, it will be easier to judge the (initial) value of the technology.

 

[Mike11]

FWIW, I'll repeat what I said in another thread. This is what I speculate Apple's PowerVR-based roadmap looks like:
0) 90nm/ARM11/MBXLite/In-House or Samsung Audio&Video (2007/2008 iPhones)
1) 65nm/ARM11/SGX520/VXD330/In-House Audio&ISP (2009 iPhone & iPhone Nano)
2) 45nm/Cortex-A9/SGX540/VXD380/VXE280/In-House Audio&ISP (2010 iPhone)

However there is still the mystery of the higher ARM clock speed for the iPod Touch 2G (which I love BTW, it does feel smoother compared to my 1G ) - I wonder what's up with that.

I'm not so sure that Apple could come up with a custom Cortex-A9/SGX54x for the iPhone in mid 2010. Cortex-A8 powered smartphones enter the market in H1/2009, so mid 2010 seems awfully early for Cortex-A9. And Apple acquired PA Semi in 2008, so that would only be 2 years for a custom design with new technology from start to device. It took TI from 2003 til 2009 to get the Cortex-A8 OMAP3 into smartphones with 45 engineers working on it. OMAP4 (Cortex-A9/SGX540) is not expected before the end of 2010 at the earliest for smartphones. And TI has a lot of experience and expertise in the custom ARM chip business...

 

[Mike11]

I'm not so sure that Apple could come up with a custom Cortex-A9/SGX54x for the iPhone in mid 2010. Cortex-A8 powered smartphones enter the market in H1/2009, so mid 2010 seems awfully early for Cortex-A9. And Apple acquired PA Semi in 2008, so that would only be 2 years for a custom design with new technology from start to device. It took TI from 2003 til 2009 to get the Cortex-A8 OMAP3 into smartphones with 45 engineers working on it. OMAP4 (Cortex-A9/SGX540) is not expected before the end of 2010 at the earliest for smartphones. And TI has a lot of experience and expertise in the custom ARM chip business...

I forgot to add: Qualcomm's Snapdragon has been a four year, $350 million project.

 

[Lazy8s]

Over a month ago, OMAP3430 product such as NEC's N-01A became available.

The theoretical specs of Cortex-A8 may seem less efficient than ARM11, but benchmarks of Pandora indicate an impressive improvement in real-world performance. I'd expect the further improvement Cortex-A9 brings will end up owing as much to the benefit of time and fab process as architectural design.

 

[Ailuros]

I think you'll find IMG has been "in" on OpenCL from the start as they are one of the co-authors of the OpenCl standard...

http://www.khronos.org/news/press/releases/the_khronos_group_releases_opencl_1.0_specification/

in which Tony King-Smith from IMG said
"Imagination is delighted to have been involved in the authoring of OpenCL, which we see as a significant development for the future of GP-GPU based computing for multimedia.”

That is true, however OpenCL was Apple's baby all along. Apple had even a patent (which I've no idea if it was filed or not) under a different name before OpenCL was announced yet unfortunately my link to it is dead by now.

Arun,

Remember the MBX Lite in the current iPhone is clocked at 50MHz it seems, while a SGX 520 on 65nm would be clocked at 150-250MHz. That seems like a very reasonable performance improvement to me if Apple doesn't increase the screen solution, which I wouldn't expect them to do before 2010.

That might open the gap between MBX Lite and 520 (and I doubt the latter would go higher than 200MHz@65nm), but the gap between a 520 and 540 would still be quite large.

Anyway if Apple is as conservative with power consumption as in the initial iPhone (which makes perfect sense if you consider that if I play a demanding game on the N95 I need to search for a power plug asap heh) you might have a point, albeit I'd easier bet on a SGX530@45nm for the third attempt then.

 

[Mike11]

That might open the gap between MBX Lite and 520 (and I doubt the latter would go higher than 200MHz@65nm), but the gap between a 520 and 540 would still be quite large.

Anyway if Apple is as conservative with power consumption as in the initial iPhone (which makes perfect sense if you consider that if I play a demanding game on the N95 I need to search for a power plug asap heh) you might have a point, albeit I'd easier bet on a SGX530@45nm for the third attempt then.

SGX530@45nm for the third iPhone attempt (2009)? I'm not so sure that 45nm is practical by Q3/2009. All the other 2009 smartphone parts seam to be in 65nm (OMAP3, Snapdragon etc.).

 

[Simon F]

That is true, however OpenCL was Apple's baby all along. Apple had even a patent (which I've no idea if it was filed or not) under a different name before OpenCL was announced yet unfortunately my link to it is dead by now.

Did you mean granted rather than filed? A patent is automatically filed the instant you present it to the patent office.

 

[Arun]

I'm not so sure that Apple could come up with a custom Cortex-A9/SGX54x for the iPhone in mid 2010. Cortex-A8 powered smartphones enter the market in H1/2009, so mid 2010 seems awfully early for Cortex-A9.

Let me answer point by point...

And Apple acquired PA Semi in 2008, so that would only be 2 years for a custom design with new technology from start to device.

Apple already had a semiconductor team in-house before PA Semi. The original iPhone SoC is not a vanilla one they bought from Samsung, but rather one they co-designed with Samsung using Samsung IP. The last page from this PDF should give you a hint of what kind of ready-made blocks are available: http://www.samsung.com/global/busin...foundry/downloads/foundry_brochure_200611.pdf

The goal with PA Semi is likely to reduce their dependence on Samsung and to ARM/PowerVR IP themselves, rather than through Samsung. They're also obviously moving away from Samsung's in-house multimedia IP, although probably not from their I/O IP. So Apple isn't new to the semiconductor game unlike what some sites are claiming; they are only moving incrementally more of the process in-house.

It took TI from 2003 til 2009 to get the Cortex-A8 OMAP3 into smartphones with 45 engineers working on it.

45 engineers working only on the CPU part, right? Obviously those engineers weren't active from 2003 to 2009, more like 2003 to 2006 and the numbers at the start were likely smaller... Anyway there are two big differences there:
- TI did a fair bit of (semi-)custom circuit work on the Cortex-A8 and they were the lead customer. Neither will likely be the case for Apple on 40nm; the A9 was made so that it would be viable to use normal synthesis tools to implement it.
- The time from tape-out to final product availability can be much lower for Apple because they can co-design hardware and software at the same time (something nobody else can do) and the chip they do in-house doesn't require baseband qualification etc. like for Nokia.

I forgot to add: Qualcomm's Snapdragon has been a four year, $350 million project.

They simply licensed the ISA and did *everything* in-house. I don't think this is comparable at all...

 

[Arun]

Yes. I can see a lot of places where the A9 in sympathetic surroundings could be a very viable alternative to x86, when binary x86 compatibility isn't a major issue.

Yup, what's most interesting with NEON is that it's very much aimed at phones too though. The FPU in Qualcomm's Snapdragon is also a 128-bit FMAC unit...

Even though computational science is my branch, you'll note my almost complete absence from the GPGPU forum. It just doesn't do anything for the problems at hand.
[...]
But most scientific/technical problems do not benefit greatly from the Big Iron, and in many cases the underlying approximations are coarse enough, or the computational load light enough, that a PC can deal with the problems so well that going further just isn't justified over the utility of doing your work on a laptop.
And I say that as part of a community that burn supercomputing cycles like there is no tomorrow.

Oh, I fully understand where you're coming from. Many problems are indeed simple enough that high-end HPC is kind of a fool's errand. But I still think the high-end HPC market will remain large enough that GPGPU will matter; and really, the problem you're describing also applies to handhelds, where most of the cool seemingly compute-intensive stuff really isn't as intensive as it might appear on first hand. There even more, GPGPU depends on finding a cool app to make it mainstream or at least to demonstrate a clear power or cost advantage as you said.

That might open the gap between MBX Lite and 520 (and I doubt the latter would go higher than 200MHz@65nm), but the gap between a 520 and 540 would still be quite large.

Why? SGX 540 is ~4x faster per MHz, isn't it? If it's also clocked 50% faster, that means 6x faster. If you go from 320x480 to 480x720 for example, the performance boost per pixel goes down to ~2.5x... That already seems much less crazy to me! As for power consumption numbers, don't confuse the core 3D numbers with what's required for the rest of the SoC including the ARM11 and other systems which might have to be online due to bad system/power gating design. SGX540@45/40nm is still a very small core, and I'd assume power to be very reasonable.

SGX530@45nm for the third iPhone attempt (2009)? I'm not so sure that 45nm is practical by Q3/2009. All the other 2009 smartphone parts seam to be in 65nm (OMAP3, Snapdragon etc.).

He meant third SoC not second so 2010+, however I only partially agree with you. TI SoCs coming out for the likes of Nokia in 2009 are likely to be in 45nm; except for those however, and maybe a few phones with Qualcomm's MSM7850 (CDMA Rev.A/B with ATI's OpenGL ES 2.0 3D core), everything in 2009 will indeed be 65nm.

As for TSMC's 40nm process, I don't think it's completely impossible that we would see a few personal media players or things like that in 4Q09 with that, given the shorter design cycles, but who knows and that depends more on execution that anything else...

 

[Mike11]

Let me answer point by point...
Apple already had a semiconductor team in-house before PA Semi. The original iPhone SoC is not a vanilla one they bought from Samsung, but rather one they co-designed with Samsung using Samsung IP. The last page from this PDF should give you a hint of what kind of ready-made blocks are available: http://www.samsung.com/global/busin...foundry/downloads/foundry_brochure_200611.pdf

The goal with PA Semi is likely to reduce their dependence on Samsung and to ARM/PowerVR IP themselves, rather than through Samsung. They're also obviously moving away from Samsung's in-house multimedia IP, although probably not from their I/O IP. So Apple isn't new to the semiconductor game unlike what some sites are claiming; they are only moving incrementally more of the process in-house.

Thanks for the clarification. Great post!

45 engineers working only on the CPU part, right? Obviously those engineers weren't active from 2003 to 2009, more like 2003 to 2006 and the numbers at the start were likely smaller... Anyway there are two big differences there:
- TI did a fair bit of (semi-)custom circuit work on the Cortex-A8 and they were the lead customer. Neither will likely be the case for Apple on 40nm; the A9 was made so that it would be viable to use normal synthesis tools to implement it.
- The time from tape-out to final product availability can be much lower for Apple because they can co-design hardware and software at the same time (something nobody else can do) and the chip they do in-house doesn't require baseband qualification etc. like for Nokia.

Good points. I didn't know the A9 required less hand-crafting then the A8 (OMAP3) for decent results.

They simply licensed the ISA and did *everything* in-house. I don't think this is comparable at all...

Doesn't P.A. Semi have a license to do the same thing? But I guess I can read between the lines that a custom ARMv7 Apple core would not be worth the effort?

He meant third SoC not second so 2010+, however I only partially agree with you. TI SoCs coming out for the likes of Nokia in 2009 are likely to be in 45nm; except for those however, and maybe a few phones with Qualcomm's MSM7850 (CDMA Rev.A/B with ATI's OpenGL ES 2.0 3D core), everything in 2009 will indeed be 65nm.

As for TSMC's 40nm process, I don't think it's completely impossible that we would see a few personal media players or things like that in 4Q09 with that, given the shorter design cycles, but who knows and that depends more on execution that anything else...

My bad, I thought he meant third device generation as in 1. iPhone, 2. iPhone 3G, 3. iPhone 09

So after all this information I'm pretty confident that a mid 2010 iPhone with higher resolution screen and Apple designed Cortex-A9 and SGX540 in 40nm is quite possible. If Apple wants to do it

 

[wco81]

Apple hasn't pushed higher resolution displays in the computer business.

If the form factor remains the same, a 3-4 inch screen, what applications would push them to use higher resolution?

 

[Mike11]

Apple hasn't pushed higher resolution displays in the computer business.

If the form factor remains the same, a 3-4 inch screen, what applications would push them to use higher resolution?

Because if they don't everyone else will have sharper displays by 2010-2011! Smartphone market leader Nokia has at least 640x360 on a smaller screen size on all their S60 touch smartphones. Blackberrys have at least the same resolution, but a smaller screen size. Windows Mobile and Android will go for 800x480 on most devices. So Apple has to upgrade at some point. Imho most likely in 2010.

Plus everything looks better with a higher resolution. And you can definitely see the difference between 480x320 and 800x480 on a 3.5" screen.

 

[Ailuros]

Resolutions on mobile devices will IMHLO scale only up to a reasonable size of the entire device. The size of the shoe isn't necessarily ergonomic or practical for a mobile phone LOL.

 

[wco81]

Just saying, their laptops, which well very well, have lower resolutions than competitors.

I'd like to see higher resolution too. Main benefit is in videos and games, perhaps. Although with higher quality screens, maybe games development becomes a more expensive proposition, unless they can easily port console games. That won't help smaller developers though.

Higher resolution in the UI or for things like Safari may not be appealing to older consumers though.

 

[Arun]

Doesn't P.A. Semi have a license to do the same thing? But I guess I can read between the lines that a custom ARMv7 Apple core would not be worth the effort?

I thought that was for the PPC ISA? Either way ARM seems to be willing to license the ARM ISA to whoever wants to take the plunge; quite different from a certain other ISA, if I may take the chance to point that out...

So after all this information I'm pretty confident that a mid 2010 iPhone with higher resolution screen and Apple designed Cortex-A9 and SGX540 in 40nm is quite possible. If Apple wants to do it

Exactly, if Apple wants to do it is a very good question i.e. how much money they want to invest into it etc. as well as how far along they were with their 65nm SoC when they bought PA Semi (remember they licensed the PowerVR cores as early as mid-2007...) - btw last quick note: Samsung doesn't have a 40nm process per-se it seems, although in terms of marketing figures it's mid-way between TSMC's 45 and 40...

I'd like to see higher resolution too. Main benefit is in videos and games, perhaps. Although with higher quality screens, maybe games development becomes a more expensive proposition, unless they can easily port console games. That won't help smaller developers though.

Yeah, resolution has also had the problem it increases the screen/backlight's power consumption. I don't have exact figures but it's definitely not negligible IIRC. In Apple's case, the big question is whether they will eventually increase the screen size to, say, 4"... If so, a larger resolution starts making even more sense.

Higher resolution in the UI or for things like Safari may not be appealing to older consumers though.

How is that relevant when you can zoom in as you see fit and the pixel size of icons etc. would likely be adjusted to remain reasonably sized? I honestly don't think it should be too much of a problem, or at least I hope it won't be; it's not quite like on a personal computer.

 

[Mike11]

Higher resolution in the UI or for things like Safari may not be appealing to older consumers though.

I also don't see the problem here. It's not like the fonts etc. get smaller, they just get sharper. For everyone who surfs and/or reads a lot on the iPhone a higher resolution screen would be a blessing.

Exactly, if Apple wants to do it is a very good question i.e. how much money they want to invest into it etc. as well as how far along they were with their 65nm SoC when they bought PA Semi (remember they licensed the PowerVR cores as early as mid-2007...) - btw last quick note: Samsung doesn't have a 40nm process per-se it seems, although in terms of marketing figures it's mid-way between TSMC's 45 and 40...

Is there some kind of exclusivity deal between Apple and Samsung till 2010 and beyond like with AT&T?

 

[Arun]

Is there some kind of exclusivity deal between Apple and Samsung till 2010 and beyond like with AT&T?

I don't think there is anything like that, but since Samsung got the manufacturing license for several SGX and VXD/VXE cores, that would simply to imply Apple plans to manufacture more than one new SoC at Samsung. That would indicate they're locked in for 65 & 45nm...

 

[wco81]

Apple is going to try to keep the form factor as small and thin as possible.

These design goals would seem to clash with bigger screens, more power hungry chips and the higher battery density needed to support these niceties.

 

[Mike11]

Apple is going to try to keep the form factor as small and thin as possible.

These design goals would seem to clash with bigger screens, more power hungry chips and the higher battery density needed to support these niceties.

I think the combined power consumption of the new chips and displays we discussed here would be in the same ballpark as todays chips/displays or even lower for comparable tasks. And all the other components become more power efficient/smaller too (broadband/storage/GPS etc.)