ICO devs discuss how they made SOTC (In FULL detail)

[mckmas8808]

They talk about everything from how they created HDR, Motion-Blur, and Fur shading to character animation, shadows, and DOF.

http://www.watch.impress.co.jp/game/docs/20051207/3dwa68.jpg
http://www.watch.impress.co.jp/game/docs/20051207/3dwa71.jpg
http://www.watch.impress.co.jp/game/docs/20051207/3dwa72.jpg
http://www.watch.impress.co.jp/game/docs/20051207/3dwa29.jpg

Devs and others can you discuss what program they are using to create this game? And can you explain to us what we are looking at in these pics? And can you also explain the pic where they are showing us the animation in the works?


Official Link for the whole article is
http://www.watch.impress.co.jp/game/docs/20051207/3dwa.htm


'one' can you please help translate?

Thanks

 

[Edge]

Thanks mckmas8808! You forgot self-shadowing. Too bad the article is not in english.

It's amazing what developers are pulling of on the PS2. Can't wait to see what this team pulls off on the PS3.

Awesome game, with very beautiful music. The music and the trailers I saw made me buy a PS2 for the first time. Have not regretted it, also now 65 hrs into Dragon Quest 8.

 

[macabre]

Nice find, I hope one can translate it for us.
The program for the models could be Lightwave 3D, you can almost read it on the top left corner.

 

[aeriic]

Nice find, I hope one can translate it for us.
The program for the models could be Lightwave 3D, you can almost read it on the top left corner.

Yep, Lightwave Modeler. Older version too...

 

[BTOA]

This was posted once here and I think the thread starter asked for one or someone to help translate it.

 

[liverkick]

This was posted once here and I think the thread starter asked for one or someone to help translate it.

That was me. It was more like a polite nudge though.

 

[rendezvous]

Since my Japanese is crappy I'll stick to guessing from the pictures.

1. Picture of motion blur. No Idea of how it is done.

2. In picture of the colossus the grey patches are for fur rendering.
As can bee seen in the picture, and even better in one picture on the page each patch contains several layers. Each layer is a slice of the fur and by stacking multiple slices you can get full strains of hair. If you don't use enough slices the will be gaps in the strains of hair as can be seen here. http://www.watch.impress.co.jp/game/docs/20051207/3dwa45.jpg

3. Looks like they are doing the animation for a colossus. The graph seems to be showing the position of the bone in the club.

4. Attack animation, wouldn't dare to guess what the graph represents.

5. Setting up running cycle, graph to the right is the height of the hips. The left tree-graph could be a representation of the bone structure.

6. Character in default position, the way it is being modelled, no transformations applied. The model is rigged (Bones added).

 

[one]

That was me. It was more like a polite nudge though.

Ah, I once checked that thread and read that lengthy interview+commentary once again done by always spot-on Mr. Zenji Nishikawa, but I had and have too little time at hand to translate it properly right now. Maybe I'll post a summary later, but if someone has technical interest in a specific area in the SOTC implementation like how they implemented the animation of the protagonist, then I'll try to focus on it. It's a very comprehensive article.

 

[AlNom]

Just wondering if you could find out the following briefly...maybe? lol

Is the HDR really HDR (by definition)? And I don't quite see why the shadows would be talked about...

 

[one]

Just wondering if you could find out the following briefly...maybe? lol

Is the HDR really HDR (by definition)? And I don't quite see why the shadows would be talked about...

No. It's pseudo HDR created by clever handwork tricks, basically the same as ICO. The map of the world has box-shaped scene definitions (they call it 'Scene Box') everywhere and when the player gets in it it shows the effects set in the Scene Box beforehand. If you are in a shrine, the outside looks glaring. These effects are set in all Scene Boxes and smoothly blended between Scene Boxes when the player moves around.

Shadows are stencil shadow volume + soft self shadow.

 

[AlNom]

Ah... thank you very much

 

[mckmas8808]

Ah, I once checked that thread and read that lengthy interview+commentary once again done by always spot-on Mr. Zenji Nishikawa, but I had and have too little time at hand to translate it properly right now. Maybe I'll post a summary later, but if someone has technical interest in a specific area in the SOTC implementation like how they implemented the animation of the protagonist, then I'll try to focus on it. It's a very comprehensive article.

Can you summarize the great animation that the characters have. I've never seen animation done on that level before.

 

[liverkick]

Ah, I once checked that thread and read that lengthy interview+commentary once again done by always spot-on Mr. Zenji Nishikawa, but I had and have too little time at hand to translate it properly right now.

Completely understood one, your contributions on here are always appreciated.

 

[centerofadmiration]

WOW. Team Ico sure is at the top of their game this early on. HDR lighting, self shadowing, all that stuff on the PS2 and they still were able to keep the game running around 20FPS during the Colossi battles? Absolutely amazing.

I still get chills thinking about the first time I laid eyes on a Colossus, and how long I kept the battle going against Colossus 5 and Colossus 13. Simply brilliant game. Now I gotta go play it again :smile:

 

[!eVo!-X Ant UK]

Use bablefish to translate it

 

[Shifty Geezer]

Is the HDR really HDR (by definition)? And I don't quite see why the shadows would be talked about...

When you say 'really HDR' what you mean is 'conventional HDR'. HDR by definition is a high dynamic range, which means a greater than 0...255 or 0...1.0 scale. Conventionally this is achieved with larger data formats, but the same effect can be achieved in other ways, such as using multiple additive textures or breaking the regular INT data structures into mantissa and exponent bits.

I don't think it's healthy for people to talk of 'real' HDR and 'fake HDR'. Everything on consoles is a fake one way or another. What matters is how the final output is produced. As long as people hold onto preconceptions as to what constitutes a feature, it's going to cause complications when different techniques are used. Just take a look at the recent NAO32 discussion, where going from an FP16 buffer to a customised 32bit per pixel colour space, parts of the internet errupted into talk of HS not having true HDR. If a game is using 'overbright' then it's using HDR of a sort. How that's implemented will determine the quality of the solution, no more and no less. Not using large FP buffers is no more 'faking' HDR then raytracing or scanline rendering is 'faking' computer graphics. They're different ways to achieve the same overall result.

 

[overclocked]

Ah, I once checked that thread and read that lengthy interview+commentary once again done by always spot-on Mr. Zenji Nishikawa, but I had and have too little time at hand to translate it properly right now. Maybe I'll post a summary later, but if someone has technical interest in a specific area in the SOTC implementation like how they implemented the animation of the protagonist, then I'll try to focus on it. It's a very comprehensive article.

Would be great if you could do that and many would enjoy it!
Kudos for the work if you get the time to make a whole summary!

 

[one]

The summary (or ended up as a loose translation dump, as it's really hard to compress) of this long interview+commentary by Zenji Nishikawa is like this:

#1 HDR and dynamic tonemapping are emulated by an alternative way

It's pseudo HDR created by clever handwork tricks, basically the same as ICO. The map of the world has box-shaped scene definitions (they call it 'Scene Box') everywhere and when the player gets in it it shows the effects predefined in the Scene Box. If you are in a shrine, the outside looks glaring.

The flow is

1. Render the far sight and save it
http://www.watch.impress.co.jp/game/docs/20051207/3dwa09.htm
2. Render the near sight
http://www.watch.impress.co.jp/game/docs/20051207/3dwa10.htm
3. Blend (1) and (2) by using Z-buffer in (2) as a mask pattern
http://www.watch.impress.co.jp/game/docs/20051207/3dwa11.htm
4. Create a mask pattern shaped by the near sight using Z-buffer in (2)
http://www.watch.impress.co.jp/game/docs/20051207/3dwa12.htm
5. Resize (4) into 64x64 by a bilinear filter
6. Blend (7) translucently in the last frame by the percentage set per Scene Box
7. Save (6) and use it for rendering the next frame
8. Enlarge (5) with a bilinear filter to an appropriate resolution for bloom effect
http://www.watch.impress.co.jp/game/docs/20051207/3dwa13.htm
9. Blend (7) and (8) translucently by the percentage set per Scene Box
http://www.watch.impress.co.jp/game/docs/20051207/3dwa14.htm

These effects are set in all Scene Boxes and smoothly blended between Scene Boxes when the player moves around and that looks like pseudo dynamic tonemapping.

This takes big human resources as you have to set those predefined scene effects in all of the huge areas in the game by hand.

#2 Motion blur, ghosting light, and frame rate

Motion blur in this game is very basic: just supersample a current frame by the movement of the camera. This camera blur is not on characters. For characters, it generates polygons from the current frame toward the positions of bones in the previous frame and applies the image of the character in the current frame to those polygons as the textures.

The ghosting light effect that appears on the occasions such as the bright sky is coming in and out of your sight is a byproduct of the pseudo HDR in #1, specifically (6) in the flow.

Motion blur is one of the predefined effects in a Scene Box.

The frame rate is variable, and it varies from 15 to 60 fps, but is alleviated by motion blur.

#3 Stencil shadow volume with self shadow

This image is specifically created for this article with a darker shadow than in-game.
http://www.watch.impress.co.jp/game/docs/20051207/3dwa34.htm
The stencil shadow volume is visualized in the second image. Shadow volume is created with VU1.
http://www.watch.impress.co.jp/game/docs/20051207/3dwa25.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa26.htm
The player, the horse, and colossi have those shadows. (In ICO, Ico and Yorda had those realtime shadows by stencil shadow volume with self-shadow)

The palyer character has 3000 polygons, and a colossus has about 18,000. Shadow models have lower polycounts to fit in the PS2 processing power. Soft shadow is done by blurring in-scene shadow in a stencil buffer, so not physically correct, but enough for the PS2 resolution.

The shadow model for the face of the player character
http://www.watch.impress.co.jp/game/docs/20051207/3dwa27.htm
The shadow model for the player character
http://www.watch.impress.co.jp/game/docs/20051207/3dwa28.htm
Superimposing the shadow model wireframe on the player character model
http://www.watch.impress.co.jp/game/docs/20051207/3dwa29.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa30.htm
Self-shadow on a colossus
http://www.watch.impress.co.jp/game/docs/20051207/3dwa31.htm
Self-shadow off
http://www.watch.impress.co.jp/game/docs/20051207/3dwa32.htm
Self-shadow on
http://www.watch.impress.co.jp/game/docs/20051207/3dwa33.htm

#4 Fur shader

As PS2 has no real fur shader capability in SM1.x, it's done by layered shells of textured polygons (3 - 6 layers).
http://www.watch.impress.co.jp/game/docs/20051207/3dwa39.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa40.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa41.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa42.htm

Also per-pixel lighting and per-pixel anisotropic lighting is not possible on PS2, but it does per-vertex lighing in these fur layers.
http://www.watch.impress.co.jp/game/docs/20051207/3dwa44.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa45.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa46.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa47.htm
5 layers
http://www.watch.impress.co.jp/game/docs/20051207/3dwa49.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa50.htm

#5 LOD for landscape, and loading

There are 3 phases in the landscape.

Far (they call it 'Super Low'): the farthest backgrounds are realtime rendered from a low-poly model into a 2D image and pasted on polygons as textures.
Middle: 600m x 600m square rendered by low fidelity landscape model
Near: 100m x 100m square rendered by high fidelity landscape model

Landmark objects in 'Super Low' areas are always on memory.

Colossi have no LOD and always use high-poly models.

The seamless background loading eliminates loading times. As this causes memory fragmentation, the resource management system checks the vertical retrace interval and does defragmentation in background when it's idle. Adresses in the program are also created on the assumption that they are remapped.

#6 Colossi as moving landscape, and deforming collision system

Colossi have low-poly collision models.

http://www.watch.impress.co.jp/game/docs/20051207/3dwa61.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa57.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa60.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa63.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa59.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa64.htm

Those collision models deform when colossi change their poses.
http://www.watch.impress.co.jp/game/docs/20051207/3dwa65.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa66.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa67.htm

When the player and a colossus collide the player is treated as a sphere. Likewise, when a colossus and a wall collides, the colossus is treated as a sphere, to save the processing power.

#7 Inverse kinematics on colossi

The realtime adaptive walking animation system with IK (Inverse Kinematics) is implemented in the colossi's animation.

Colossus AI is represented as reactions to the player entering particular zones. Those reactions are not created by programmers but by designers with a collosi scripting software.
http://www.watch.impress.co.jp/game/docs/20051207/3dwa73.htm

When a colossus walks, it calculates the direction and distance of movement from the AI. The basic walking animation is designed by hand, and footsteps are adjusted to the actual landscape by IK. The player character and the horse also get scene feedback by IK.
http://www.watch.impress.co.jp/game/docs/20051207/3dwa68.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa69.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa70.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa71.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa72.htm

#8 Physics motion and Add Motion

When the player sticks to a colossus, he is shaken by physics simulation of 2 pendulums for the player's hand-to-bust and waist-to-foot (multiple pendulum simulation). Besides spring pendulum simulation is applied to elastic hands.

However, only with those simulations it became inorganic, so they added human motion designed by hand to the result of this simulation. They call it "Add Motion" system.

But even with those 2 factors some glitchs occured. Then they added IK to them and finally created the natural motion made of 3 synthesized factors which can create actions that both programmers and designers have never seen before. This system is used not only against the player but in the motion of the horse on which the player is riding on.

"Add Motion" in action
http://www.watch.impress.co.jp/game/docs/20051207/3dwa76.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa77.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa93.htm
Pendulum and deforming collision model
http://www.watch.impress.co.jp/game/docs/20051207/3dwa67.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa94.htm
Pendulum and Add Motion
http://www.watch.impress.co.jp/game/docs/20051207/3dwa78.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa79.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa80.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa81.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa82.htm
http://www.watch.impress.co.jp/game/docs/20051207/3dwa83.htm

IK for the horse:
Basic pose
http://www.watch.impress.co.jp/game/docs/20051207/3dwa87.htm
Simple 2-bone IK, not natural
http://www.watch.impress.co.jp/game/docs/20051207/3dwa86.htm
The release version
http://www.watch.impress.co.jp/game/docs/20051207/3dwa85.htm

#9 Pseudo volume particle and pseudo light diffusion

Normal particle with a plain texture
http://www.watch.impress.co.jp/game/docs/20051207/3dwa95.gif
http://www.watch.impress.co.jp/game/docs/20051207/3dwa96.gif
http://www.watch.impress.co.jp/game/docs/20051207/3dwa99.gif
http://www.watch.impress.co.jp/game/docs/20051207/3dwa101.gif
Pseudo volume particle by per-vertex lighting
http://www.watch.impress.co.jp/game/docs/20051207/3dwa97.gif
http://www.watch.impress.co.jp/game/docs/20051207/3dwa98.gif
http://www.watch.impress.co.jp/game/docs/20051207/3dwa100.gif
http://www.watch.impress.co.jp/game/docs/20051207/3dwa102.gif

They also created a pseudo light diffusion shader (they call it "Fairy Shader") by simple anisotropic lighting.
It's tuned by designers' hand.
http://www.watch.impress.co.jp/game/docs/20051207/3dwa91.htm
Without Fairy Shader
http://www.watch.impress.co.jp/game/docs/20051207/3dwa88.htm
Fairy Shader 30%
http://www.watch.impress.co.jp/game/docs/20051207/3dwa89.htm
Fairy Shader 100%
http://www.watch.impress.co.jp/game/docs/20051207/3dwa90.htm

The created what they call "Velvet Shader" for the skin of the horse.
Normal specular map
http://www.watch.impress.co.jp/game/docs/20051207/3dwa111.gif
http://www.watch.impress.co.jp/game/docs/20051207/3dwa112.gif
Velvet Shader
http://www.watch.impress.co.jp/game/docs/20051207/3dwa113.gif
http://www.watch.impress.co.jp/game/docs/20051207/3dwa114.gif

#10 Final words

According to Fumito Ueda, they started the development of SoTC 3 years ago. They reached the fantastic visual style of SoTC via try and err, just like in ICO. The expressions used in SoTC, such as fur, motion blur, Add Motion, Fairy Shader and Velvet Shader were first examined on LightWave by Ueda as the technology inventor, then were suggested to each different division in the team. For both ICO and SoTC, the visual styles were not only created with advanced technologies of visual programmers but also with visual plans by designers and other aspects such as the good artwork and craftsmanship.

Nishikawa asks why they didn't move the project from PS2 to PS3. They answer that it'd take 3 years or more and it's good that it's on PS2 as the image of "a bit beyond PS2" is one of the advantages of SoTC.

For the next project for PS3

Mainstream games released so far are about the same scale point-vs-point collision like sword-vs-sword. But for the next-generation, collision against a large surface like SoTC will be a common theme. Though I can comment nothing concrete on the next project, it's possible in some form we may use the theme of the large-scale "hold-onto action" we actualized in SoTC. In some way, it will be meaningful to realize these things that can be a theme for a next-generation game in the current generation in advance... it's one of the reasons why we completed this work on PS2.

 

[dubert]

Since my Japanese is crappy I'll stick to guessing from the pictures.
1. Picture of motion blur. No Idea of how it is done.

It seems they are using the camera movement direction to generate the motion blur. Translating polygons mapped with the raw image, using layers of quads, and rendering those transparent over eachother. This technique doesn't account for object deformations, only camera movement. I could be wrong tho, since I don't read Japanese.
But I know someone how can, if you guys want me to point him to that page...

 

[mckmas8808]

Thanks "one". I shall let the world view your great deeds. We shall really claim you as the ultimate poster.