PS3 as external accelerator - Fixstars CodecSys CE10 'faster than reatime' encoder

I didn't think this was for "most people". I thought this was only for professionals and prosumers that would be dealing with HD material. Did I misunderstand that?

 

I am completely underwhelmed with the results so far. At high bitrates (16mbps) the quality is poor on any high motion material. No two-pass encoding is a basic issue IMHO.

I will give 3mbps a go though with a challenging 720p30 stream... you'll be able to stream that in the Eurogamer HD player!

From a compressionist's POV, the "ever-changing codec" argument doesn't really cut much mustard and certainly from everything I have seen so far, you are far more likely to get better results from a pro perspective by spending the $2,000 USD on a fast i7 PC.

 

Er no. All the testing has been done on a PS3. You can even try it yourself.

As an update I've done a 720p30 encode at 3mbps and it's bloody awful. The scary thing is that CodecSys's definition of 3mbps appears to be somewhat different to x264 and not in a good way: 27.7MB vs 35.1MB.

 

If you're going to budget 150Mbps, you'd be far, far better off using an intermediate codec such as CineForm HD, Avid DNxHD or ProRes. For a start you'd be moving to a 4:2:2 colourspace versus AVC's 4:2:0. With a decent h264 encoder at 1080p24, you will see massively diminishing returns on anything above 30Mbps.

 

Are you testing the Personal version ? It seems different from the Pro one:

Personal: http://codecsys.fixstars.com/images/support/tutorial/setting.png

Pro:http://codecsys.fixstars.com/images/products/ce10/CE10Setting.png

There is an "Advanced Video Setting" tab on the pro version. May be I should try myself too.

As for bitrate, Blu-ray supports up to 40Mbps. If it's part of the pipeline, perhaps they'd need the H.264 source to go beyond 30Mbps.

 

I see. No wonder you guys are b*tching about it. Websites and personal viewing have little use for super high bitrate encodes too.

Would still be interesting to do a 3Mbps vs a 100-150Mbps run @ 1080p.

What is the output spec of the 3Mbps file ? And also time to produce one on x264 ? If they are comparable, 27.7Mb vs 35.1Mb is a plus. Otherwise, where did they cut corners ?


EDIT: Is the install easy ? I don't want to mess too much with it.

Btw, here's their road map:
http://codecsys.fixstars.com/en/support/ce10/releases.html

 

x264 can encode 1080p at 24fps using one core of a 3.0GHz Core 2 Duo. In real-time. The thing is of course that the quality will be absolutely dreadful. Not useable. So you up the quality settings and get a good encode. But it takes longer.

The thing with CodecSys is that the quality settings (bandwidth aside) are essentially set in stone. I can't make them any better. And they're still producing very poor results. I could of course keep lowering the quality of x264 until the speed tests get within ballpark of CodecSys, but what's the point when the results are unusable? The whole point of h264 is to get more video information in fewer bytes, but it is not giving me that.

My 720p30 encode takes about four minutes on a £50 Intel E5200 2.5GHz Pentium dual core CPU. A much higher quality version on the i7 takes three minutes. This still lags behind the time taken to encode the same file on CodecSys, which is about 52 seconds. But the point is obvious: I can use the x264 encodes. I can't use the CodecSys one because it doesn't look anywhere near good enough.

So for the cost of a $200 CodecSys license, I could either upgrade my PC with a new CPU, RAM and motherboard and get results that are actually good, or I can use my PS3 to make poor encodes. Not a difficult choice.

In terms of the $2,000 "Professional" option, the bottom line is that the results do not warrant the price tag. I could buy a mighty i7 workstation for that price and again get better results and probably faster too since i7 is already being used with x264 for real time 1080p24 HD encoding.

Bottom line is that the total lack of flexibility is the problem here. Speed is of course important, but compare the options available in CodecSys to x264 and there's simply no contest. The fact that CodecSys offers no two-pass encoding and no choice in motion estimation algorithms essentially means that it is fundamentally limited.

 

The lack of flexibility is probably because they need to conform to Blu-ray specs, and also because they are still working on the improvements (This is a v1.1 product according to the Japanese translation).

As for worth, I am curious if the super high bandwidth was a necesseity for Blu-ray authoring. What happens if you need to encode a full length movie for authoring to Blu-ray ? I sent Codecsys the URL of this thread. Don't know if they are interested to come here and clarify. The sales people might not be bothered.

Small companies are usually more careful in resource allocation. I'd be surprised if CE-10 is only a fluff product.

 

Eh? Blu-ray specs conform perfectly with multi-pass encoding. You can also encode at whatever bitrate you want with BD... in fact it's essential for the BD-9 format. There is no requirement for super-high use of bandwidth. Bottom line is that in a straight like for like challenge on identical bitrates, x264 significantly outperforms CE-10, even when both are set to their respective Blu-ray encoding profiles, and even when x264 is forced to work on average bitrate alone over one pass.

Of course, you get better quality at higher bandwidth simply through brute-forcing the data through. Any video encoder will produce better results at lower settings the more bandwidth you give it. But the point is you can scale x264 down if you want, but you can't scale CE-10 up.

One small thing worthy of clarification. The larger file size appears to be down to the use of the .ts transport stream container. I remuxed into an MP4, and the file size looks right now.

With regards the quality level, it is worth pointing that that x264 has been proven to outperform the CUDA-based encoding solutions too, so the results here should not be too surprising. The fact that CE-10 is v1.1 cuts little mustard when you're comparing a $200/$2000 product with a free one.

 

Yes, but it's necessary to support 40Mbps H.264 + 5.1 LPCM audio for Blu-ray. Hence, the high bit rates. The deal with single pass encoding using Codecsys's algorithm is to save time according to their marketing literature. As you mentioned, there are already many 2 pass encoders on the market. They'd try to appeal to a certain segment of users.

I believe H.264 supports a fixed/prefined set of profiles and levels. The encoders will need to be aware of them. In fact, v1.1 is not Baseline Profile compliant. According to their road map, it will be in v1.2.

They would have done their research vis-a-vis other Blu-ray authoring solution. Perhaps they feel that PS3 + CE-10 is a good match against tying up a workstation for long hours.

 

Sorry but the notion of a $2,000 pro-level encoder without even the option of two-pass encoding is quite staggering. Remember that the first pass is basically a scan of the video to determine where best to allocate bandwidth budget. In the case of x264 it adds about 25% to 30% to the total encode time. If CE-10 is so fast, can't it do the first pass faster too? Surely the whole point of this is - as you say - to act as an accelerator. Not to impose a "one size fits all" encoding profile that puts speed way ahead of quality.

Now, I'm sure there are things you could use this encoder for. Non-challenging video will look good. But why spend $200/$2000 on an encoder that can't handle the stuff that h.264 was designed to handle well?

 

If I read it correctly, CodecSys's patent essentially says they could determine the best codec and settings to use on-the-fly: http://www.codecsys.com/cmsdocuments/BI_CanadianPatentAllowance_03182009FINAL.pdf

 

Well I'll do some more tests today, but I'm not sure what else there is to try. I am running at maximum quality settings and the results are disappointing. All I can do now is dumb down x264 to make it faster, but what's the point when the quality isn't good enough?

I can run some SSIM perceptual quality tests on the encodes if you really need further convincing but just to the human eye it's obvious that this is a fast, but less efficient encoder that offers significantly lower functionality than any pro-level AVC encoder I've ever used.

 

I'll see if I can find someone from Fixstar to clarify. Leaving us in this state is not helping either.

 

Yes, that's why I was probing the input source used (how long ?), the encoding duration, size, and quality/bitrate. The algorithm they used may be useful only in a particular context. If their target customers (not us !) don't see the benefit, the product will die. Or they will change strategy.

 

In many ways, I am the target audience. I have an i7 workstation set up that almost exclusively encodes h264. Even if CE-10 was "only" twice as fast, I would be all over it.

I had an expert look at CE-10's output and he was genuinely surprised by the lack of quality in the stream, citing an emphasis on intra blocks (where most of the bandwidth has gone), poor motion search that struggles to even make the most of short-range motion vectors, a complete lack of adaptive B frame placement, no sub 16x16 b-modes and sudden bandwidth drop-outs. Plus other stuff too that is perhaps too boring and technical to get into here.

Looking at the settings available, he agreed that there simply isn't much room to manouevre in getting a better quality stream. The settings available are the most limited of any pro-level encoder I've ever used. Mainconcept Reference gets a lot of grief for being over-expensive, but it's almost god-like in terms of flexibility and performance compared to this.

All of which can be side-lined by throwing more bandwidth at the problem and all of which kinda misses the point of working with h264 in the first place - that being that it is currently the most effective way to transmit video with limited bandwidth. If you want to dump out a sustained 40mbps Blu-ray stream while you pop out for lunch, then it has its uses. But you'd be better off buying an i7 workstation for the same money and saving yourself the cost of a PS3 into the bargain.

Unless you're pumping out multiple Blu-rays 24/7, any kind of pro-level authoring studio would be going for bandwidth efficient, quality encoding.

The argument seems to be here that nobody would put out a poor product when the stuff already out there has set a standard, but that's just not the way the world works. If it did, every FPS would be a CoD4. CE-10 is more of a Turning Point.

 

its a shame but as already said it seems this is more a transitional product rather than a really serious one.

i really wanted to say good things about it, and i can to a degree....but not even to a prosumer grade...end users today may find it just about usable for a few days though for MS 360 and ps3 use...and maybe even 640x480, 512x384, 512x288P use in the lower bandwidth settings ?

its better than badaboom, and AVIVO HW Encoders on the quality and speed side,
as its got real ref=3 High profile L4.0 and L4.1 settings by default, were these other two dont even come close.

the main site goes in to how your harddrives and LAN network really matter to keep the Encoding speed up, but after reading this thread it seems these are not a real problem, that problem being the same fixed far to low settings that come with BB and the Envivo apps, and thats a shame.

the specs sheet they do say "*Encoding speed decreases if the speed of reading the hard disk or the Ethernet transfer rate is slow. However this has no impact on the encoded image quality. "

and in the http://codecsys.fixstars.com/en/support/ce10/faq/howto_use.html#q5003
"...The encoding speed seems to be slow. Why?
In CE-10, when the PS3 is used as the accelerator for the H.264 encoding process, a full HD (1920x1080) image can be encoded into 22-23 frames per second. CE-10 achieves high speed processing by transferring each frame data through Ethernet and compressing them in PS.

Therefore, if intended performance is not achieved, the following reasons may be considered:
HDD read capability on PC is not sufficient.
Your Ethernet adapter only supports 100base-T.
PC and PLAYSTAION3 are connected via a network HUB.

In order to realize the high speed encoding and maximize the power of the PS3, you have to use a high speed part as the HDD where movie files are stored or install a RAID-0 storage system and connect the PC and PS3 directly to Gigabit Ethernet using crossover cables.
The read speed of movie files during encoding appears in [Input image read speed (average)] of the status bar; if this value is low, review the items above.
..."


im a little puzzled, today SquallMX posted some small test he did, see this thread for the full pictures and links to the final mkvs
http://forum.doom9.org/showthread.php?p=1306613#post1306613
" I made a little test using CE-10 Pro Trial:

Source: Blu-ray Promo from Gran Torino BR disc
Details: VC-1 1080p 23.976 fps 13 Mbps (Max 23 Mbps)
Decoder: NVTools + AviSynth (Hardware Accelerated)
Laptop Specs: Intel Core2Duo T5800, GeForce 9300M, Windows 7 Build 7260 x64
Others: Resized to 1440x1080p using Spline36

pics

Desired Bitrate: 5000 Kbps
Real Bitrate: 4250 Kbps (Aprox)
Encoding Speed: 12 fps (Aprox)
Total Encoding Time: 3:13.150

x264 core 68 r1183M f21daff 1-Pass ABR (Blu-ray Compliant)
Settings:

Code:
--profile high --level 4.1 --keyint 48 --min-keyint 4 --direct auto --deblock -2:-1 --psy-rd 0.8:0.2 --partitions p8x8,b8x8,i4x4,i8x8 --qpmax 40 --ipratio 1.1 --pbratio 1.2 --vbv-bufsize 15000 --vbv-maxrate 15000 --thread-input --aq-strength 0.8 --ssim --output "J:\Gran Torino\BDMV\STREAM\1440.mkv" "J:\Gran Torino\BDMV\STREAM\1440.avs" --mvrange 511 --aud --nal-hrd --sar 4:3Desired Bitrate: 5000 Kbps
Real Bitrate: 4675 Kbps
Encoding Speed: 5.39 fps
Total Encoding Time: 7:37.000 (Aprox)

Comparative Stills:

pics

Full Promo:
CE-10
x264

Conclusions on CE-10:
1.- Only half "Real Time" speed.
2.- Laptop's CPU at 90% (Using NVTools for Decoding).
3.- Only 2.2x faster than x264 using mid range CPU for laptop, with mid settings (Desktops QuadCore are probably faster than CE-10).
4.- Lower Quality than x264
"

he says he gets 2.2% faster than x264 with these setting above on his Intel Core2Duo T5800, GeForce 9300M, but it seem people testing in this thread didnt manage that ?

also he got me thinking, sadly iv not kept up with the PPC/Altivec/cell x264/ffmpeg optimisations and patchs so i dont know if they when compiled and run on PS3 compare with the current x86 version speeds as the devs on on doom9 keep x86 updated....

but how does PPC/Altivec/cell x264 compare speed/quality wise with the current x86 x264 version above ?

clearly x86 x264 is still THE AVC Pro ENCODER Everyone needs to beat for quality and speed when run on the right high end CPUs etc, but does a current PPC based x264 run directly on the PS3 with YDL6.2 also make CE-10 look bad?

im trying to work out why they also set the so called pro version at 150Mbit/s limit, the only thing i can think of is the H264 Levels, but perhaps im reading that wrong ?
http://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

level4.2 seems to have a 150Mbit limit for Hi10P
1920×1080@64.0 (4)
2048×1080@60.0 (4)

but wouldnt pro's willing to pay US$1,999 only consider that IF there were also High, High 10, High 4:2:2, and High 4:4:4 profiles Hi422P, Hi444PP ,and
November 2007, the JVT was working on an extension of H.264/AVC towards scalability by an Annex (G) called Scalable Video Coding (SVC)containing Scalable Baseline, Scalable High, and Scalable High Intra profiles.

all in all, it's nice to have a 14 day full trial for trying some basic high profile, higher bitrate AVC encoding on your idel PS3 , but i cant see it being werth paying the US$199 asking price, and im thnking a PPC/Altivec/Cell SIMD compiled x264 might get close or better this speed and far better quality anyway with the generic Blu-ray Compliant line above!

 

another thing , they say this

http://codecsys.fixstars.com/en/support/ce10/tutorial/bluray.html
Prepare source video

Prepare video at either resolution of 720x480, 1280x720, 1440x1080, 1920x1080 in supported format. Current version of CodecSys CE-10 does not support resolution resizing, so that output video resolution is supposed to be same as input.

http://codecsys.fixstars.com/en/support/ce10/manuals/3_common_videoformat.html
"Input video format setting
Set up input video format.

Avairable format types are following


YUV (YUV file with YUV4:2:0 8bit planer format)
AVI (*1)
MPEG2 (MPEG2 file *1) "



*1. Directshow filter has to be installed properly to import AVI file and MPEG2 file.
See also : Install Directshow filter "

and their online Encoder
https://codecsys.fixstars.com/online-trial/en/trial/upload.html

"...Supported file type is only YUV 4:2:0 Planar (IYUV/I420) file.
Upload file size can be up to 1.5GB..."

says https://codecsys.fixstars.com/online-trial/en/trial/howto_make_yuv/
"...
ffmpeg -i video.avi -pix_fmt yuv420p video.yuv
...
ffmpeg -i video.avi -pix_fmt yuv420p -t 10.0 -s 1280x720 video.yuv
....
etc"

did anyone try and encode some lower 7Mbit/s 25p/50p, ref=2/3, H@L4.0/@L4.1 720x480, 720x576, 1280x720, 1440x1080, as well as 1920x1080p video clips....for the tests.