PS3 as external accelerator - Fixstars CodecSys CE10 'faster than reatime' encoder

[Shifty Geezer]

Just chatting with a friend about encoding which he's getting into, and looking up Cell development, I found FixStars are releasing an encoder, the CodecSys CE-10 (http://us.fixstars.com/products/codecsys/CE10_USletter.pdf), that runs on PS3 connected to PC as an accelerator over Gigabit. They reckon 1.2x realtime speed at Full HD! Most importantly it means someone's using PS3 as an external accelerator 'board'. This paves the way for other applications to use PS3 as such (my dream of PS3 as a softsynth lives on!)

[Brad Grenz]

Yeah, it's a pretty cool idea. It sounds like a pretty strait forward use of linux on PS3 for server side processing of data sent from a client over the network. There shouldn't be anything stopping people from rolling similar solutions for other kinds of work that would benefit from SPE acceleration.

[one]

They are planning to sell non-professional version of the said encoder later.
For something like this to work, PS3's price and size should get steady reduction.

[Brad Grenz]

Professionals can buy addin card or blade server versions of their encoding solution. I don't think the PS3's price and size are much of an issue with this product. It's just a creative use of something you might already have. It's not targeted at creating a server farm. The faster than realtime encodes are achieved on a single PS3. If you are at the "prosumer" level authoring Blu-rays a PS3 would be a good thing to have for testing and playback of your discs anyway. And I dare say the folks at Digital Foundry might find some utility in such a product. We'll see what the price of the software actually is and what license it is released under before we might say it's worth buying for someone who wants to rip and recode their movies for DLNA purposes.

[Shifty Geezer]

They are planning to sell non-professional version of the said encoder later.What exactly do you mean by non-professional. Isn't this a home-enthusiast product, with professionals being targeted with the more convenient add-in Cell board? I'm expecting a pricetag that's competitive with a big CPU or GPU for those who are otherwise looking to upgrade their PC to accelerate encoding.

[Arnold Beckenbauer]

Just chatting with a friend about encoding which he's getting into, and looking up Cell development, I found FixStars are releasing an encoder, the CodecSys CE-10 (http://us.fixstars.com/products/codecsys/CE10_USletter.pdf), that runs on PS3 connected to PC as an accelerator over Gigabit. They reckon 1.2x realtime speed at Full HD! Most importantly it means someone's using PS3 as an external accelerator 'board'. This paves the way for other applications to use PS3 as such (my dream of PS3 as a softsynth lives on!)

Is it really fast? You get with Power Director 7 realtime encoding with one GTX260 and a dual-core CPU. And Badaoom is extremely fast, but its quality isn't good.

http://www.fixstars.com/en/products/codecsys/sample.html

The following are sample videos which are encoded by CodecSys CE-100. Please use these videos for image quality evaluation
The quality is really amazing.

What exactly do you mean by non-professional. Isn't this a home-enthusiast product, with professionals being targeted with the more convenient add-in Cell board? I'm expecting a pricetag that's competitive with a big CPU or GPU for those who are otherwise looking to upgrade their PC to accelerate encoding.
Non-professional could mean: it's made for AVCHD.

[one]

What exactly do you mean by non-professional. Isn't this a home-enthusiast product, with professionals being targeted with the more convenient add-in Cell board? I'm expecting a pricetag that's competitive with a big CPU or GPU for those who are otherwise looking to upgrade their PC to accelerate encoding.At its Japanese page it retains "Professional" in its product name.
http://www.fixstars.com/products/codecsys/ce10.html

According to this report about NAB 2009 (the professional broadcast equipments expo in the US), the rep said, the professional version would be offered as subscription-based. Around the same timeframe in June, they will also release a consumer-package version at a "reasonable price" with some file-format restrictions.

http://av.watch.impress.co.jp/docs/series/zooma/20090424_153779.html

[Shifty Geezer]

Is it really fast? You get with Power Director 7 realtime encoding with one GTX260.Do you have a link? I haven't seen encoding figures apart from some poorly defiend examples like CyberLink's nVidia page (http://www.cyberlink.com/products/powerdirector/cuda-optimization_en_US.html) and a weak reference to a 50% decrease in time when CUDA is applied to Premier CS4 through RapiHD (http://www.synbio.org.uk/scientific-computing-news/1248.html).

But there's also the matter of cost here, that for people who already have a PS3, it's (possibly!) cheaper to get a software package to use that than buy an expensive GPU just for encoding (assuming no use besides that. Don't really need a GTX260 for browsing the web ;)). PS3's encoding is certainly fast, even if not the fastest option available.

[one]

It's now on sale for the Japanese market.

http://codecsys.fixstars.com/ja/index.html

CE-10 Personal - 15,840 yen ($164) for a 1-year license
CE-10 Professional - 158,400 yen ($1,640) for a 1-year license (24,800 yen for a 1-month license)

Personal can encode movies up to 15Mbps, while it's up to 150Mbps for Professional.
No commercial encoding is allowed for Personal. All other properties are the same for the both versions.

[patsu]

Cool. 14 day free trial here: https://codecsys.fixstars.com/ja/trial.html

[Shifty Geezer]

Ooooooo. Not too pricey at all. Any chance Grandmaster can give it an analysis? 15 megabit seems plenty good enough for many home rippers, but is that really true, or will quality suffer? A free trial is excellent. When's the English rollout?!!

[V3]

15 Megabit is about broadcast quality. It's pretty good. If you are working any higher you're probably a professional.

[Arnold Beckenbauer]

There are some words about CE-10 in the german magazine c't: So they reached almost realtime encoding, it supports uncompressed AVI, YUV and MPEG2. The CE-10 personal costs $200 per year, CE-10 professional $ 2000. No word about quality.

[Shifty Geezer]

Is there any word about an NA/EU release date?

[Lucid_Dreamer]

I'm late on this. I didn't know this encodes at about the same speed as CUDA BadaBOOM with a GTX-285 and almost twice as fast as an Intel Core i7 965XE. That's extremely interesting.

[pjbliverpool]

I'm late on this. I didn't know this encodes at about the same speed as CUDA BadaBOOM with a GTX-285 and almost twice as fast as an Intel Core i7 965XE. That's extremely interesting.

Quality can vary dramatically between implementations though. Comparisons are on very dodgy ground unless your pretty much using the same encoder on both platforms. Or at the very least, using identical quality settings.

[Shifty Geezer]

Indeed. A proper comparison needs quality as well and encode times to be evaluated.

Edit : I note the '1.2x' figure isn't 'faster than realtime' but 'slower then', so though accelerated, 1080p resolution hasn't hit realtime at the consumer level yet.

[Lucid_Dreamer]

Quality can vary dramatically between implementations though. Comparisons are on very dodgy ground unless your pretty much using the same encoder on both platforms. Or at the very least, using identical quality settings.
Further up in the posts, the quality was referred to as "amazing". Has that changed now? If so, why would they lower the quality? Also, why would one need to use the same encoder on both platforms to determine a quality difference between two results? I'm puzzled.

Edit : I note the '1.2x' figure isn't 'faster than realtime' but 'slower then', so though accelerated, 1080p resolution hasn't hit realtime at the consumer level yet.
I thought 1x is realtime in this case. Wouldn't that mean anything over 1x is faster and no slower than realtime?

[Shifty Geezer]

I think the 1.2x refers to the time taken, not speed. That is, it's not "Encoding speed is 1.2x realtime" but "encoding time taken is only 1.2x realtime". So if the film is 1 hour long, it'll take about 1 hour 12 minutes to encode. The figures I've seen show a 1080p encode taking longer than the source by a small amount, and the CE-10 introductory paragraph reads
...making it the most accessible, near real-time product...
720p and 480p are much faster. Encoding for portable will be very quick!

[grandmaster]

I'm trying it now. Initial impressions are that it is... basic. I know they want to be "realtime" but no two-pass encoding? Yikes.

It's using around 33% of a 2.5GHz mobile Penryn and it's processing at 8fps - probably down to the fact I'm running in a lossless 720p60 encode from a USB drive onto my laptop. That said, I just set a small clip on a copy run across to the internal drive and the average FPS in CodecSys remains unaltered at 8fps, so maybe not.

h264 encoding options are limited. Results i'm getting are excellent but it's Resident Evil 5 (easy to compress) and it's using an average of 16mbps... even on a crappy h264 encoder (hello Quicktime!) you'll get "amazing" results ;)

It is quite neat to use though - the hookup with PS3 is seamless. The PS3 side of things appears to be built on YDL. It does support the codecs I use for HD capture, meaning it's a straight VFW or DS pipeline on the video decoding side of things.

Will try to do lots more with it today and built it up into a DF blog entry. A comparison with x264 is natural but I suppose CUDA based solutions on my i7/GTX295 would be good too ;)


Quick Update: Nope, 720p60 is 8fps even on a clip hosted locally in the ultra-fast-to-decode CineForm HD codec. I'll see if I can snaffle another trial licence and try it on the i7 with uncompressed video.

[pjbliverpool]

Further up in the posts, the quality was referred to as "amazing". Has that changed now? If so, why would they lower the quality? Also, why would one need to use the same encoder on both platforms to determine a quality difference between two results? I'm puzzled.

Thats hardly a quantitive measure though is it? Unless your encoding the exact same source material to the exact same end quality, how is a comparison going to be valid?

I'm suggesting that to see which hardware is faster, you would ideally use the same encoder on each. In practical terms though thats probably not going to be possible since the software will be targeted specifically for the architecture of the hardware its running on. So the next best thing is simply to take the same source material and ensure you are using the same quality settings to encode it.

[Jawed]

So the next best thing is simply to take the same source material and ensure you are using the same quality settings to encode it.
Two different versions of x264 (e.g. I've just compared builds 1130 and 1173*), given the same input file and using the same encoder settings, will not necessarily produce the same file.

So, you are then in "what does it mean when I say the image quality is the same?" territory. That's a very taxing question, to which PSNR and SSIM are no answer.

Jawed

* - 1173 is the last generally available build of x264 before the changes in defaults and introduction of presets+profiles. 1173 was built just before the end of June and 1130 dates from March-ish.

[Shifty Geezer]

Quick Update: Nope, 720p60 is 8fps even on a clip hosted locally in the ultra-fast-to-decode CineForm HD codec. I'll see if I can snaffle another trial licence and try it on the i7 with uncompressed video.Well that's a long way off realtime! I eagerly anticipate your findings.

[grandmaster]

Moved onto the i7, decoded the AVI to uncompressed YV12 (ie the same pixel format as the final encode) all in an effort to eliminate any decoding overhead. Frame rate from a 720p60 stream is now a very respectable 30fps. I'll see if there are any other tricks I can use to beef that up.

CPU usage on the i7 is about 7%, and it was about 33% on the Penryn laptop. This disparity is probably down to the decoding.

Using multiple slices gives me an FPS boost (this is where the picture is literally divided into four slices encoded/decoded in parallel). I can up motion detection quality from the default 88 to 100 and still get up to 32fps.

[grandmaster]

I get 1080p encoding at around 20-24fps. Weirdly - and rather disturbingly - the quality settings don't appear to impact performance that much.

[Lucid_Dreamer]

Thats hardly a quantitive measure though is it? Unless your encoding the exact same source material to the exact same end quality, how is a comparison going to be valid?

So the next best thing is simply to take the same source material and ensure you are using the same quality settings to encode it.
The article said they weren't doing that somewhere? How else would it be a comparison, if these things weren't considered?

[pjbliverpool]

The article said they weren't doing that somewhere? How else would it be a comparison, if these things weren't considered?

Maybe I missed it but I haven't seen any direct comparisons between Badaboom, i7 and CE10 yet.

[patsu]

I get 1080p encoding at around 20-24fps. Weirdly - and rather disturbingly - the quality settings don't appear to impact performance that much.

According to CodecSys' press release...


CodecSys Technology

CodecSys utilizes a patented, multi-codec approach in which a video stream is
analyzed and the codec best-suited for a particular frame or video sequence is
automatically selected from an entire library of specialized codecs. By
compressing video to under 3Mbps, CodecSys enables video providers to reduce
their bandwidth needs by more than 50 percent for HD quality and pack more video
into less bandwidth.


Perhaps you should test it at low bitrate to see what the fuzz is all about. Since they are positioning this as a "real-time" encoder, the CodecSys' algorithm + single pass may be more suitable for their needs. We have other free/affordable solutions for 2-pass encoding anyway.

Also, are you testing the free trial professional edition or the consumer version ? Is it gimped in anyway compared to the full professional version ?


EDIT: Found Professional vs Personal edition comparison here: http://codecsys.fixstars.com/en/ce10/specs.html

The free trial version provides full functionality. The Professional version supports up to 150Mbps encoding. Does it still encode at 20-24fps for 1080p, 150Mbps ?

[Shifty Geezer]

Word from my mate who's trying this, it was encoding VOB's ripped from DVDs at the same speed as his dual-core Pentium. It certainly wasn't churning out h.264 encodes at a rate of knots.

I should get to see it first hand tomorrow.

[patsu]

Word from my mate who's trying this, it was encoding VOB's ripped from DVDs at the same speed as his dual-core Pentium. It certainly wasn't churning out h.264 encodes at a rate of knots.

I should get to see it first hand tomorrow.

Copy the VOB from DVD to HDD and try. Also, if the ripping does not involve re-encoding, it may not exercise H.264 encoding (for the Pentium case). What was the final output ?

[Shifty Geezer]

The VOB is on the PC HDD, already ripped. Output was whatever the default settings are!

[patsu]

DVD doesn't use H.264, so it's up to the ripping + transcoding software's configuration. Check it ! ^_^

[Shifty Geezer]

DVD doesn't use H.264Now you've confused me! I thought that was the whole point. Here is a video feed in goodness knows whatever format it is in. CE-10 turns the video feed into h.264 to view on h.264 devices.

Now I think about it, is the bottleneck here the decoding on the PC? It's decoding the MPEG into a video feed that CE-10 encodes to h.264, so no matter how fast CE-10 may be at shrinking that to PSP resolution, say, it'll always run at PC speed. Whereas if the source was a BRD already in h.264, CE-10 could shrink it to a small PSP sized movie very quickly. :???: What happens to the VOB data? Is that passed bytewise to the PS3 to decode and encode, or is it decoded on the PC and that data passed for encoding? :???: :???:

Coming from 2D image manipulation, this is all voodoo to me! Load in picture. Save as other format. Easy as pie! Mostly because every application suuports nigh on every format.

[patsu]

Ha ha... what I meant was DVD uses MPEG2. I don't know if the PC ripping software transcodes the video into H.264 at all (or may be it transcode it into something else). CE-10 will transcode it into H.264 (because it's the only thing it knows).

If PC is the bottleneck, then it may not be CPU bound because grandmaster tried it on 2 laptops with different CPUs.

[Shifty Geezer]

CE-10 will transcode it into H.264 (because it's the only thing it knows).If true, the point to CE-10 seems kinda negligable. It's okay if you're mastering digital srouces, but for ripping DVDs and such, if the decoding is done on the PC, you're still going to be running very slow. I expected it'd allow you to rip video media and scale it for different storage and displays, and to do all this quickly using the SPEs.

[ShaidarHaran]

Maybe I'm misunderstanding but isn't the point of this application to use a PS3 to accelerate the encode process and all the testing so far has been without the use of a PS3?

[patsu]

If true, the point to CE-10 seems kinda negligable. It's okay if you're mastering digital srouces, but for ripping DVDs and such, if the decoding is done on the PC, you're still going to be running very slow. I expected it'd allow you to rip video media and scale it for different storage and displays, and to do all this quickly using the SPEs.

*Shrug* I don't know. According to the results here, the encoding duration changes depending on the bitrate: http://codecsys.fixstars.com/en/ce10/gallery.html. The decoding time on PC may depend on movie content. But yes, something like SPE/SPUREngine should help more.

EDIT: Ah... I see what grandmaster was testing now. His result does not include decoding an MPEG 2 stream on i7 though (Uncompressed input took 7% of i7 ?).

[patsu]

Anyone testing with lower than 3Mbps and higher than 15Mbps encodes ? Those are the cases where CodecSys' algorithm kicks in (or shines). Otherwise, I suspect it's "standard" SPU stuff.

[Shifty Geezer]

I got to see this yesterday. Overall impression - what's the point? We tried an SDTV h.264 video from a camcorder and encoded it via CE-10. It ran a little slower than realtime. We tried several options, and all were the same uniform speed. By comparison, the dual core AMD running Handbrake was only a little slower. So CE-10's not an insanely fast encoder by any stretch. Perhaps it can encode full HD faster than any other options, but that's not what most people want. If it can't shrink movie rips any faster than a PC, why bother?

[patsu]

What bitrates did you end up with ?

[Lucid_Dreamer]

I got to see this yesterday. Overall impression - what's the point? We tried an SDTV h.264 video from a camcorder and encoded it via CE-10. It ran a little slower than realtime. We tried several options, and all were the same uniform speed. By comparison, the dual core AMD running Handbrake was only a little slower. So CE-10's not an insanely fast encoder by any stretch. Perhaps it can encode full HD faster than any other options, but that's not what most people want. If it can't shrink movie rips any faster than a PC, why bother?
I didn't think this was for "most people". I thought this was only for professionals and prosumers that would be dealing with HD material. Did I misunderstand that?

[grandmaster]

I am completely underwhelmed with the results so far. At high bitrates (16mbps) the quality is poor on any high motion material. No two-pass encoding is a basic issue IMHO.

I will give 3mbps a go though with a challenging 720p30 stream... you'll be able to stream that in the Eurogamer HD player!

From a compressionist's POV, the "ever-changing codec" argument doesn't really cut much mustard and certainly from everything I have seen so far, you are far more likely to get better results from a pro perspective by spending the $2,000 USD on a fast i7 PC.

[grandmaster]

Maybe I'm misunderstanding but isn't the point of this application to use a PS3 to accelerate the encode process and all the testing so far has been without the use of a PS3?

Er no. All the testing has been done on a PS3. You can even try it yourself.

As an update I've done a 720p30 encode at 3mbps and it's bloody awful. The scary thing is that CodecSys's definition of 3mbps appears to be somewhat different to x264 and not in a good way: 27.7MB vs 35.1MB.

[grandmaster]

The free trial version provides full functionality. The Professional version supports up to 150Mbps encoding. Does it still encode at 20-24fps for 1080p, 150Mbps ?

If you're going to budget 150Mbps, you'd be far, far better off using an intermediate codec such as CineForm HD, Avid DNxHD or ProRes. For a start you'd be moving to a 4:2:2 colourspace versus AVC's 4:2:0. With a decent h264 encoder at 1080p24, you will see massively diminishing returns on anything above 30Mbps.

[patsu]

Are you testing the Personal version ? It seems different from the Pro one:

Personal: http://codecsys.fixstars.com/images/support/tutorial/setting.png

Pro:http://codecsys.fixstars.com/images/products/ce10/CE10Setting.png

There is an "Advanced Video Setting" tab on the pro version. May be I should try myself too.

As for bitrate, Blu-ray supports up to 40Mbps. If it's part of the pipeline, perhaps they'd need the H.264 source to go beyond 30Mbps.

[Shifty Geezer]

There's only one trial version to download, the Pro.

[patsu]

I see. No wonder you guys are b*tching about it. Websites and personal viewing have little use for super high bitrate encodes too.

Would still be interesting to do a 3Mbps vs a 100-150Mbps run @ 1080p.

As an update I've done a 720p30 encode at 3mbps and it's bloody awful. The scary thing is that CodecSys's definition of 3mbps appears to be somewhat different to x264 and not in a good way: 27.7MB vs 35.1MB.

What is the output spec of the 3Mbps file ? And also time to produce one on x264 ? If they are comparable, 27.7Mb vs 35.1Mb is a plus. Otherwise, where did they cut corners ?


EDIT: Is the install easy ? I don't want to mess too much with it.

Btw, here's their road map:
http://codecsys.fixstars.com/en/support/ce10/releases.html

[grandmaster]

x264 can encode 1080p at 24fps using one core of a 3.0GHz Core 2 Duo. In real-time. The thing is of course that the quality will be absolutely dreadful. Not useable. So you up the quality settings and get a good encode. But it takes longer.

The thing with CodecSys is that the quality settings (bandwidth aside) are essentially set in stone. I can't make them any better. And they're still producing very poor results. I could of course keep lowering the quality of x264 until the speed tests get within ballpark of CodecSys, but what's the point when the results are unusable? The whole point of h264 is to get more video information in fewer bytes, but it is not giving me that.

My 720p30 encode takes about four minutes on a £50 Intel E5200 2.5GHz Pentium dual core CPU. A much higher quality version on the i7 takes three minutes. This still lags behind the time taken to encode the same file on CodecSys, which is about 52 seconds. But the point is obvious: I can use the x264 encodes. I can't use the CodecSys one because it doesn't look anywhere near good enough.

So for the cost of a $200 CodecSys license, I could either upgrade my PC with a new CPU, RAM and motherboard and get results that are actually good, or I can use my PS3 to make poor encodes. Not a difficult choice.

In terms of the $2,000 "Professional" option, the bottom line is that the results do not warrant the price tag. I could buy a mighty i7 workstation for that price and again get better results and probably faster too since i7 is already being used with x264 for real time 1080p24 HD encoding.

Bottom line is that the total lack of flexibility is the problem here. Speed is of course important, but compare the options available in CodecSys to x264 and there's simply no contest. The fact that CodecSys offers no two-pass encoding and no choice in motion estimation algorithms essentially means that it is fundamentally limited.

[patsu]

The lack of flexibility is probably because they need to conform to Blu-ray specs, and also because they are still working on the improvements (This is a v1.1 product according to the Japanese translation).

As for worth, I am curious if the super high bandwidth was a necesseity for Blu-ray authoring. What happens if you need to encode a full length movie for authoring to Blu-ray ? I sent Codecsys the URL of this thread. Don't know if they are interested to come here and clarify. The sales people might not be bothered.

Small companies are usually more careful in resource allocation. I'd be surprised if CE-10 is only a fluff product.

[grandmaster]

Eh? Blu-ray specs conform perfectly with multi-pass encoding. You can also encode at whatever bitrate you want with BD... in fact it's essential for the BD-9 format. There is no requirement for super-high use of bandwidth. Bottom line is that in a straight like for like challenge on identical bitrates, x264 significantly outperforms CE-10, even when both are set to their respective Blu-ray encoding profiles, and even when x264 is forced to work on average bitrate alone over one pass.

Of course, you get better quality at higher bandwidth simply through brute-forcing the data through. Any video encoder will produce better results at lower settings the more bandwidth you give it. But the point is you can scale x264 down if you want, but you can't scale CE-10 up.

One small thing worthy of clarification. The larger file size appears to be down to the use of the .ts transport stream container. I remuxed into an MP4, and the file size looks right now.

With regards the quality level, it is worth pointing that that x264 has been proven to outperform the CUDA-based encoding solutions too, so the results here should not be too surprising. The fact that CE-10 is v1.1 cuts little mustard when you're comparing a $200/$2000 product with a free one.

[patsu]

Yes, but it's necessary to support 40Mbps H.264 + 5.1 LPCM audio for Blu-ray. Hence, the high bit rates. The deal with single pass encoding using Codecsys's algorithm is to save time according to their marketing literature. As you mentioned, there are already many 2 pass encoders on the market. They'd try to appeal to a certain segment of users.

I believe H.264 supports a fixed/prefined set of profiles and levels. The encoders will need to be aware of them. In fact, v1.1 is not Baseline Profile compliant. According to their road map, it will be in v1.2.

With regards the quality level, it is worth pointing that that x264 has been proven to outperform the CUDA-based encoding solutions too, so the results here should not be too surprising. The fact that CE-10 is v1.1 cuts little mustard when you're comparing a $200/$2000 product with a free one.

They would have done their research vis-a-vis other Blu-ray authoring solution. Perhaps they feel that PS3 + CE-10 is a good match against tying up a workstation for long hours.

[grandmaster]

Sorry but the notion of a $2,000 pro-level encoder without even the option of two-pass encoding is quite staggering. Remember that the first pass is basically a scan of the video to determine where best to allocate bandwidth budget. In the case of x264 it adds about 25% to 30% to the total encode time. If CE-10 is so fast, can't it do the first pass faster too? Surely the whole point of this is - as you say - to act as an accelerator. Not to impose a "one size fits all" encoding profile that puts speed way ahead of quality.

Now, I'm sure there are things you could use this encoder for. Non-challenging video will look good. But why spend $200/$2000 on an encoder that can't handle the stuff that h.264 was designed to handle well?

[patsu]

If I read it correctly, CodecSys's patent essentially says they could determine the best codec and settings to use on-the-fly: http://www.codecsys.com/cmsdocuments/BI_CanadianPatentAllowance_03182009FINAL.pdf

[grandmaster]

Well I'll do some more tests today, but I'm not sure what else there is to try. I am running at maximum quality settings and the results are disappointing. All I can do now is dumb down x264 to make it faster, but what's the point when the quality isn't good enough?

I can run some SSIM perceptual quality tests on the encodes if you really need further convincing but just to the human eye it's obvious that this is a fast, but less efficient encoder that offers significantly lower functionality than any pro-level AVC encoder I've ever used.

[patsu]

I'll see if I can find someone from Fixstar to clarify. Leaving us in this state is not helping either.

[Shifty Geezer]

If I read it correctly, CodecSys's patent essentially says they could determine the best codec and settings to use on-the-fly: http://www.codecsys.com/cmsdocuments/BI_CanadianPatentAllowance_03182009FINAL.pdf
Yet people using it see poor results and not particularly fast encode times. I dunno about you but I'll take user experience over marketing gumpf and patents and day of the week ;)

[patsu]

Yes, that's why I was probing the input source used (how long ?), the encoding duration, size, and quality/bitrate. The algorithm they used may be useful only in a particular context. If their target customers (not us !) don't see the benefit, the product will die. Or they will change strategy.

[grandmaster]

In many ways, I am the target audience. I have an i7 workstation set up that almost exclusively encodes h264. Even if CE-10 was "only" twice as fast, I would be all over it.

I had an expert look at CE-10's output and he was genuinely surprised by the lack of quality in the stream, citing an emphasis on intra blocks (where most of the bandwidth has gone), poor motion search that struggles to even make the most of short-range motion vectors, a complete lack of adaptive B frame placement, no sub 16x16 b-modes and sudden bandwidth drop-outs. Plus other stuff too that is perhaps too boring and technical to get into here.

Looking at the settings available, he agreed that there simply isn't much room to manouevre in getting a better quality stream. The settings available are the most limited of any pro-level encoder I've ever used. Mainconcept Reference gets a lot of grief for being over-expensive, but it's almost god-like in terms of flexibility and performance compared to this.

All of which can be side-lined by throwing more bandwidth at the problem and all of which kinda misses the point of working with h264 in the first place - that being that it is currently the most effective way to transmit video with limited bandwidth. If you want to dump out a sustained 40mbps Blu-ray stream while you pop out for lunch, then it has its uses. But you'd be better off buying an i7 workstation for the same money and saving yourself the cost of a PS3 into the bargain.

Unless you're pumping out multiple Blu-rays 24/7, any kind of pro-level authoring studio would be going for bandwidth efficient, quality encoding.

The argument seems to be here that nobody would put out a poor product when the stuff already out there has set a standard, but that's just not the way the world works. If it did, every FPS would be a CoD4. CE-10 is more of a Turning Point.

[popper]

its a shame but as already said it seems this is more a transitional product rather than a really serious one.

i really wanted to say good things about it, and i can to a degree....but not even to a prosumer grade...end users today may find it just about usable for a few days though for MS 360 and ps3 use...and maybe even 640x480, 512x384, 512x288P use in the lower bandwidth settings ?

its better than badaboom, and AVIVO HW Encoders on the quality and speed side,
as its got real ref=3 High profile L4.0 and L4.1 settings by default, were these other two dont even come close.

the main site goes in to how your harddrives and LAN network really matter to keep the Encoding speed up, but after reading this thread it seems these are not a real problem, that problem being the same fixed far to low settings that come with BB and the Envivo apps, and thats a shame.

the specs sheet they do say "*Encoding speed decreases if the speed of reading the hard disk or the Ethernet transfer rate is slow. However this has no impact on the encoded image quality. "

and in the http://codecsys.fixstars.com/en/support/ce10/faq/howto_use.html#q5003
"...The encoding speed seems to be slow. Why?
In CE-10, when the PS3 is used as the accelerator for the H.264 encoding process, a full HD (1920x1080) image can be encoded into 22-23 frames per second. CE-10 achieves high speed processing by transferring each frame data through Ethernet and compressing them in PS.

Therefore, if intended performance is not achieved, the following reasons may be considered:
HDD read capability on PC is not sufficient.
Your Ethernet adapter only supports 100base-T.
PC and PLAYSTAION3 are connected via a network HUB.

In order to realize the high speed encoding and maximize the power of the PS3, you have to use a high speed part as the HDD where movie files are stored or install a RAID-0 storage system and connect the PC and PS3 directly to Gigabit Ethernet using crossover cables.
The read speed of movie files during encoding appears in [Input image read speed (average)] of the status bar; if this value is low, review the items above.
..."


im a little puzzled, today SquallMX posted some small test he did, see this thread for the full pictures and links to the final mkvs
http://forum.doom9.org/showthread.php?p=1306613#post1306613
" I made a little test using CE-10 Pro Trial:

Source: Blu-ray Promo from Gran Torino BR disc
Details: VC-1 1080p 23.976 fps 13 Mbps (Max 23 Mbps)
Decoder: NVTools + AviSynth (Hardware Accelerated)
Laptop Specs: Intel Core2Duo T5800, GeForce 9300M, Windows 7 Build 7260 x64
Others: Resized to 1440x1080p using Spline36

pics

Desired Bitrate: 5000 Kbps
Real Bitrate: 4250 Kbps (Aprox)
Encoding Speed: 12 fps (Aprox)
Total Encoding Time: 3:13.150

x264 core 68 r1183M f21daff 1-Pass ABR (Blu-ray Compliant)
Settings:

Code:
--profile high --level 4.1 --keyint 48 --min-keyint 4 --direct auto --deblock -2:-1 --psy-rd 0.8:0.2 --partitions p8x8,b8x8,i4x4,i8x8 --qpmax 40 --ipratio 1.1 --pbratio 1.2 --vbv-bufsize 15000 --vbv-maxrate 15000 --thread-input --aq-strength 0.8 --ssim --output "J:\Gran Torino\BDMV\STREAM\1440.mkv" "J:\Gran Torino\BDMV\STREAM\1440.avs" --mvrange 511 --aud --nal-hrd --sar 4:3Desired Bitrate: 5000 Kbps
Real Bitrate: 4675 Kbps
Encoding Speed: 5.39 fps
Total Encoding Time: 7:37.000 (Aprox)

Comparative Stills:

pics

Full Promo:
CE-10
x264

Conclusions on CE-10:
1.- Only half "Real Time" speed.
2.- Laptop's CPU at 90% (Using NVTools for Decoding).
3.- Only 2.2x faster than x264 using mid range CPU for laptop, with mid settings (Desktops QuadCore are probably faster than CE-10).
4.- Lower Quality than x264
"

he says he gets 2.2% faster than x264 with these setting above on his Intel Core2Duo T5800, GeForce 9300M, but it seem people testing in this thread didnt manage that ?

also he got me thinking, sadly iv not kept up with the PPC/Altivec/cell x264/ffmpeg optimisations and patchs so i dont know if they when compiled and run on PS3 compare with the current x86 version speeds as the devs on on doom9 keep x86 updated....

but how does PPC/Altivec/cell x264 compare speed/quality wise with the current x86 x264 version above ?

clearly x86 x264 is still THE AVC Pro ENCODER Everyone needs to beat for quality and speed when run on the right high end CPUs etc, but does a current PPC based x264 run directly on the PS3 with YDL6.2 also make CE-10 look bad?

im trying to work out why they also set the so called pro version at 150Mbit/s limit, the only thing i can think of is the H264 Levels, but perhaps im reading that wrong ?
http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

level4.2 seems to have a 150Mbit limit for Hi10P
1920×1080@64.0 (4)
2048×1080@60.0 (4)

but wouldnt pro's willing to pay US$1,999 only consider that IF there were also High, High 10, High 4:2:2, and High 4:4:4 profiles Hi422P, Hi444PP ,and
November 2007, the JVT was working on an extension of H.264/AVC towards scalability by an Annex (G) called Scalable Video Coding (SVC)containing Scalable Baseline, Scalable High, and Scalable High Intra profiles.

all in all, it's nice to have a 14 day full trial for trying some basic high profile, higher bitrate AVC encoding on your idel PS3 , but i cant see it being werth paying the US$199 asking price, and im thnking a PPC/Altivec/Cell SIMD compiled x264 might get close or better this speed and far better quality anyway with the generic Blu-ray Compliant line above!

[popper]

another thing , they say this

http://codecsys.fixstars.com/en/support/ce10/tutorial/bluray.html
Prepare source video

Prepare video at either resolution of 720x480, 1280x720, 1440x1080, 1920x1080 in supported format. Current version of CodecSys CE-10 does not support resolution resizing, so that output video resolution is supposed to be same as input.

http://codecsys.fixstars.com/en/support/ce10/manuals/3_common_videoformat.html
"Input video format setting
Set up input video format.

Avairable format types are following


YUV (YUV file with YUV4:2:0 8bit planer format)
AVI (*1)
MPEG2 (MPEG2 file *1) "



*1. Directshow filter has to be installed properly to import AVI file and MPEG2 file.
See also : Install Directshow filter "

and their online Encoder
https://codecsys.fixstars.com/online-trial/en/trial/upload.html

"...Supported file type is only YUV 4:2:0 Planar (IYUV/I420) file.
Upload file size can be up to 1.5GB..."

says https://codecsys.fixstars.com/online-trial/en/trial/howto_make_yuv/
"...
ffmpeg -i video.avi -pix_fmt yuv420p video.yuv
...
ffmpeg -i video.avi -pix_fmt yuv420p -t 10.0 -s 1280x720 video.yuv
....
etc"

did anyone try and encode some lower 7Mbit/s 25p/50p, ref=2/3, H@L4.0/@L4.1 720x480, 720x576, 1280x720, 1440x1080, as well as 1920x1080p video clips....for the tests.

[Lucid_Dreamer]

Were the previous tests done by grandmaster using a Gigabit connection, etc?

[Arnold Beckenbauer]

http://www.eurogamer.net/articles/codecsys-video-encoding-on-ps3-article

It's actually not the first SPE-based encoder. Leadtek has its PCI Express-based SpursEngine card that works very quickly indeed using four SPUs, but doesn't appear to offer much in the way of quality compared to competitors, according to this review. ...

There are special de- and encoders for MPEG2 and H.264, so the four SPEs are used for upscaling, filtering etc.

[patsu]

he says he gets 2.2% faster than x264 with these setting above on his Intel Core2Duo T5800, GeForce 9300M, but it seem people testing in this thread didnt manage that ?


It's 2.2 times faster compared to a mid-range laptop CPU (single pass), although he predicts that QuadCore Desktop PC may be faster. He didn't measure 2-pass since CE-10 doesn't support one. What is the power consumption of a QuadCore ?


Conclusions on CE-10:
1.- Only half "Real Time" speed.
2.- Laptop's CPU at 90% (Using NVTools for Decoding).
3.- Only 2.2x faster than x264 using mid range CPU for laptop, with mid settings (Desktops QuadCore are probably faster than CE-10).
4.- Lower Quality than x264


The direct gigabit network and RAID 0 requirements may explain the bottleneck people encountered earlier.



All of which can be side-lined by throwing more bandwidth at the problem and all of which kinda misses the point of working with h264 in the first place - that being that it is currently the most effective way to transmit video with limited bandwidth. If you want to dump out a sustained 40mbps Blu-ray stream while you pop out for lunch, then it has its uses. But you'd be better off buying an i7 workstation for the same money and saving yourself the cost of a PS3 into the bargain.

Unless you're pumping out multiple Blu-rays 24/7, any kind of pro-level authoring studio would be going for bandwidth efficient, quality encoding.

im trying to work out why they also set the so called pro version at 150Mbit/s limit, the only thing i can think of is the H264 Levels, but perhaps im reading that wrong ?
http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

level4.2 seems to have a 150Mbit limit for Hi10P
1920×1080@64.0 (4)
2048×1080@60.0 (4)

The marketing literature claimed level 5 and 5.1 support. Isn't level 5 closer to 150Mbps ? [EDIT: Okay, you mentioned Hi10P profile for level 4.2]
At that higher level, will the relative performance difference -- if any -- be even wider ?

The same material compares CE-10 to pure hardware encoders. That's probably the reference they are measuring against (due to the limited options in CE-10 at the moment).


EDIT: Also found a full software, real-time H.264 encoder by Sony:
http://pro.sony.com/bbsccms/assets/files/mkt/broadcast/brochures/blucode_broch.pdf

It runs on QuadCore PC, maxed out at 40Mbps, and can be distributed to 4 workstations for the highest picture quality. Seems to support lot's of nice features.

I wonder if FixStar can adopt the same approach: Distribute the workload to 4 or more PS3s (at a fraction of the cost and power consumption) to obtain higher picture quality.

[Neb]

Why would you need RAID0?

My 1 year old WD 640GB SATA 2 HDD does read/write speeds of *60-80+MB/480-640+Mbit* or write and read on the same disk at the same time *35-40MB/280-320Mbit*. CPU usage barely registers.


*Practical usage measurement during copy/read or both operations in Vista of folders etc.

[patsu]

May be from excessive seeks and contention between disk operations during the encoding process (if the encoder divides its work into 4 slices, the HDD will get a lot of overlapping read operations in different "regions").
Your measurement above indicates that doing overlapped disk operations (read and write together) lowers the performance significantly. It probably also depends on what kind of HDD tech was used in other people's tests.

[pcflynn89]

review of the encoder compared to x264 on core i7

http://www.eurogamer.net/articles/codecsys-video-encoding-on-ps3-article

[patsu]

I believe said article is written by grandmaster.


it runs from PC but with all the hard processing actually performed by the Cell chip in your PlayStation 3, which is connected to your computer via a conventional LAN cable.

The CE-10 FAQ page specifies a gigabit network with RAID 0 array. The user won't get the highest performance using a 100Mb LAN.

[JPT]

How big are the files that are going the be encoded? Small enough to put into a ramdisk on the PC?

[Arwin]

I believe said article is written by grandmaster.

The CE-10 FAQ page specifies a gigabit network with RAID 0 array. The user won't get the highest performance using a 100Mb LAN.

Well it's not going to make the encoding better or make it double pass I reckon, so it won't matter much as it stands.

[patsu]

It's a v1.1 product. As long as the foundation is solid, they will improve:
http://codecsys.fixstars.com/en/support/ce10/releases.html

August 2009 ver1.2 Encoder core upgrade
...

September 2009 ver1.3 Encoder core upgrade
...

October 2009 ver1.4 Encoder core upgrade
...

However if the tester/reviewer does not set up the use environment properly, it will be problematic in each and every case (because the bottleneck looks like the network and disk access). I am not saying CE-10 is a great product. I am saying it has its position. The 2-pass encoding may not be the objective for them at this point because they want to sell on the speed (save 30% time base on software tricks alone). If they can improve the quality further, then it will become more valuable. Otherwise, it'd just fold, or they can introduce 2-pass later (e.g., after they are done with the basic encoder core).

In a startup business like this, it is not uncommon to establish a beachhead based on a trait first (In this case, speed) even though other areas may fall behind. If they can do up to 150Mbps, it may also mean they can go into the fringe of digital cinema domain (e.g., 4k x 2k @ 30fps). If you look at how their web pages evolve, they have broadened their service to product customization too. As long as they can get enough business based on the platform, we should be able to see better stuff coming out.

As it stands right now, yes, the basic product lacks in encoder quality, but they'd be able to use the professional/integration services to fund further development of the core. This is also part of the reason I don't think grandmaster is the target audience. They can't sell CE-10 for $199 or even $1999 and hope to survive (plus requirements like gigabit network and RAID 0 array hinder casual adoption). They are after bigger fish with more complex integration and custom needs.

Also, source code for CE-10:
http://codecsys.fixstars.com/en/support/ce10/source.html