The pros, cons, and techniques of procedural asset creation (renamed)

[3dcgi]

Yeah, computer assisted art. What is the definition of procedural art ?

Is Uncharted's animation blending considered procedural ? The artists create the basic parts. The computer weave them together in real-time.

In BF3, they build occlusion culling and other tech because it's not feasible for the artists to hand create destruction for every buildings.

If we continue to have bigger and more complex environment, automation may be a necessary evil.

Procedural is a broad term and might mean different things to different people. I don't know about Uncharted's animation blending, but even if it's simple keyframe interpolation I guess you could consider it to be procedural. This technique is so old/common that I don't think people consider it when talking about procedural stuff.

Procedural often means not hand painted or placed. Consider a procedural wood or marble texture. It can be controlled by an artist, but isn't painted.

 

[Shifty Geezer]

Right, but that has been attempted for quite some time now.

We haven't had displacement shaders until now, nor even much CPU power to toy with. Now we have got them, there are possibilities. And before Laa-Yosh says people won't look at all varied so there's no point in even trying, there's a lot more in games than just people!

Shifty, there's a difference between a tool that generates stuff on its own - and a tool that can help an artist by taking over the tasks that can be automated, but it still relies on a lot of human input.

Have I said otherwise?

LOTR crowd scenes and a lot of other stuff is based on vast amounts of manual work.

You know, you aren't answering any of my questions. Did the procedural aspect of Massive eliminate a lot of work that would have made LOTR's crowd scenes impossible or too expensive? If the answer is 'yes' then it shows how procedural content can take the artist's base work and automate its construction into a busy crowd, proving procedural art assists are good. If the answer is 'no', then WETA made the biggest mistake of their lives and should have just employed artists to manually create the fight scenes, along with everyone who's ever used Massive since.

You can generate dirt maps similar to how ambient occlusion works, but a good artist will also move in and paint over this mask and then add variety to the dirt layer to create a pleasing result. The procedural dirtmap on its own will remain to look artificial.

You're thinking too small and focused. Take a large texture of stones, bigger than those stone steps. On the top row show the top left corner of this stone map. The next step, show some other part of the texture. The next step, another part. The artists still draw a stone texture following all artistic rules, but we get variation on the steps. If the artists want the centre area of the steps worn down from long use (artistic consideration beyond the scope of most games), then a shader could reduce the presence of the stone texture in the areas that are worn. It may not be an approach that'd hold up to full artistic scrutiny, but few things in games ever do! We haven't abolished all these new GI hacks because they aren't accurate, nor given up on reflection maps because they are technically bunk and just resigned ourselves to only matte/gloss surfaces.

Or see this one:
http://www.cgarchitect.com/upclose/article1_CW.asp

Why do you keep going back to the 'artists are still involved' point, when I have never, ever said otherwise? Artists create the building blocks, and automated systems construct worlds out of them, or mix them up a bit to add variety. And this link proves the value of procedurally assisted art! Clearly it was better use of the budget to have artists create the building blocks for the CityBot to assemble, than for those artists to model 90,000 buildings from scratch. Chris White even says it's impossible - "Knowing that it would be impossible to hand paint these thousands of buildings, we created a special building shader that would make the process easier." So projecting this to Infamous 3/4/5, if we want a city made of varied buildings and not clones as we have now, is not the solution just like this CityBot system, to have artists create the components that can be procedurally assembled?

So it's not like, hey, we have a procedural city generator that we can buy in the software store...

Right. Now we have the processing power, isn't it about time people started creating these tools? They've already got procedural tools for simple tasks like SpeedTree. Next up would be SpeedTown.

Another example is something simple - T-shirts in a crowd...

I agree with that, and yes, it'd stand out. But then you'd change the look if you wanted to prevent that. You'd either choose an art direction that loses the T-shirt, or have enough components that everything can look varied. There are lots of examples of digital 'paper-dolls' where a user chooses their avatar from pieces. Heck, let's pick Mii's. We already have procedural crowd creation on Wii of all things by piecing together a photofit of components. Same with Home. 20 pieces of clothing that can be mixed and match takes up less RAM and offers more variety than 20 predefined models. Now let's look at this from the other direction. You're in charge of the art for next-gen Infamous. Sucker Punch say to you they want varied people characters in their city and not all looking like clones as last gen. How are you going to approach that? Are you going to say it's plain impossible as it'd require too much work, or suggest they spend 3/4 of their budget just modeling thousands of people, or look into a system that'll enable a variety of people to be pieced together from bits?

I can't seem to get across the idea that this doesn't have to be a perfect solution, and that you're viewpoint (it never going to create convincing people is proof that procedural content is a dead end not worth pursuing) is coming at it from the wrong direction. Look at what progress has been made so far. Look at how automated tools can improve things. Then we need to figure out how to implement and progress these techniques to solve budget issues, fit in with finite console hardware, while also improving the visuals we have in game. This has never been about replacing artists with one-button solutions that create whole worlds, but your position seems to be challenging that idea alone and not looking at how we can improve the efficiency of creating art on a budget and with serious hardware constraints.

 

[MfA]

Also lets not forget what is the alternative if you want large worlds ... Dragon Age 2 ...

 

[orangpelupa]

what if procedurally generated content that still "overseen" by the graphic artist?
like in architect where using a program called "Grasshopper" to "procedurally" create many alternative designs in short time.

btw for furniture position inside a room, its usually follow certain aspect. in asia environment:
like in bedroom - have window facing east, then near that is a bed.
in the theory its called "form follow function/needs" sorry if that not the right english sentence.

so maybe in game, that have furniture already created manually. The furniture placing can be procedurally generated by following "form follow function" and "Feng-shui" parameter. Like when an interior designer is thinking to place the furniture, following that rules.

 

[Laa-Yosh]

What I'm saying is that procedural content isn't going to be enough on its own, in most cases the help one can get from such systems is minimal. You'll still have to create insane amounts of building blocks, spend a lot of time to direct the Massive crowds and so on.

Whereas there are other approaches that can easily prove to work better. Rage and it's unique virtual texturing is a superior solution to the texture machinations you suggest, for example.

Also, the problem with these tools isn't lack of processing power but the inherent stupidity of the computer. You can't recreate a human's abilities, you can only automate repetitive, mundane tasks and that is the biggest problem here.

 

[Laa-Yosh]

Lots of games get away with relatively mundane surroundings ... I'll ask again, Witcher 2 huts (ie. the vast majority of buildings) ... doable by procedural generation or not?

I'm sorry, I've never played any Witcher games, so I have no idea at all.

 

[Jedi2016]

Shifty, there's a difference between a tool that generates stuff on its own - and a tool that can help an artist by taking over the tasks that can be automated, but it still relies on a lot of human input.

Very true.

My own experience with procedural tools is pretty limited. When I first started working in CGI as a hobby, procedural textures were cool, but I quickly learned just how limiting they really were. In the time it would take me to tweak a procedural noise texture to look more or less how I wanted, I could go into Photoshop and just hand-scribble something random that would end up looking a lot better.

I've also used some tools that generate various content in various ways.. tree builders, instancing tools, etc, and no matter how "random" you tell it to be, it never really is. It all looks like it's been procedurally generated, which is its greatest downside. Yeah, I can push a button and build an entire city in an instant, but it looks so computery-fake that I can't actually use it in anything.

Do these tools have their place? Sure. But we're nowhere near the point of being able to procedurally generate an entire world, and honestly, I don't think we ever will be.

 

[Laa-Yosh]

Here's a somewhat related article...

http://dresdencodak.tumblr.com/page/5

vs.

Changing hairdos and trying small displacements won't register as different characters. Sure these drawings are exaggerated but it's the same for realistic characters too. Reusing the same head with slight modifications has never worked, using 4-5 separate heads has always been better.


So, regarding your question about building characters for Infamous, I have no straight answer because too much is unknown, but I can give you some more examples on how we've dealt with character variations previously.

The restrictions are the most important, especially the budget: how many man-months can we spend on it. Also, what are the skill levels of the people involved, what are the render time limitations (we have such, too). What are the rigging and animation related limitations (use of cloth simulation, hair and so on).


For the original Assassin's Creed 2 intro we needed to build a lot of carnival dancers.
We had a single male and female body type to speed up the simulation and most of the variety was in the robe types and headwear/masks, then we expaned it using the textures (the lighting and shaders emphasized the patterns on the fabrics).
We had a randomizer tool, but the way we ended up using it was to render a sequence of all the resulting combinations, then the art director went through it and marked the ones that looked good enough and were okay to use. The completely random combinations looked pretty bad sometimes.

For a recent project that involved, well, zombies, we skipped the randomizing completely. We had the same skeleton, three body types, three kinds of upper/lower/footwear, a few types of hair and a cap. Each torso mesh had 3 head variations using a blendshape and a set of displacement/normal maps (we were unable to go with separate head meshes). The faces didn't turn out to be as good as we've hoped, by the way, partially because we couldn't change eye and mouth positions as the eyeball/teeth geometry couldn't be moved. Still we had to be careful with the various hair/cap and head combinations.
We've then created a large character sheet in Photoshop and manually designed about 20 fixed variations, named each of them and went on with no randomization/proceduralism at all. So it was like character A has body type 1, head type 2, t-shirt with texture 1, jeans with texture 3 and so on. We've found that it worked far better as we could fit the styles of the various cloth pieces better, you wouldn't wear the pants of a suit with a T-shirt for example. Combined with the females we had about 30 unique characters which doesn't sound much but it was enough and none of them looked ridiculous.

We thought about adding stuff like zombified policemen/firemen, but that's a very complex issue. These kinds of characters stand out from a crowd in several ways - you immediately notice a cop among civilians because of the uniform, so you have very limited reuse of this asset. It also separates the crowd into civilians and the other kind, so to get him to blend better, you'll have to add other characters with such stand-out features (firemen, construction worker, supermarket sales crew, waiter etc etc). So you have to build a lot more of these assets that still have very, very limited use - so this is something you can only do if you have a very large budget.
Then there's the question of height and age which is very similar. One kid will stand out, and unfortunately multiple kids will only work if they're not all the same age and height. But proportions change very distinctively with age: small kids are 4 head-lenght tall, an adult is 7-8 head-length tall, so you'd also have to cover several variations inbetween. Each age group also needs a new skeleton and new sets of animations, and it's important to accept that you can't create these procedurally, so once again it requires an extension of the budget. Oh, and kid's clothes are different, you don't want to see old people wearing young people's clothing, and so on and so on.

So this is why we ended up with about 30 people for the crowd, it was enough for our purposes and they didn't look that much different to the 4 'hero' characters in the movie (but different enough to still register as less detailed, unfortunately).
If we had to create a much larger cast with the same budget and tech restrictions, we would have had to go with some procedural tool but it would've looked a lot worse. It'd also require a more serious tool development, so that we could restrict the combinations using rules; and probably come up with a method to fit clothing and acessories to the various body types more automatically so that the artists can concentrate on building the base assets (but this is definitely more complicated for characters with fully simulated clothing, compared to today's games where everything is just skinned to a skeleton).

 

[RudeCurve]

I've noticed that more and more games have really good character creation tools built into the game itself. It's similar to the built-in "level creation" tools in games too. There must be a way to take these creation engine functions and make a standalone tool with it so you could export the models into other games.

A game with really good character creation function is Smash Court Tennis 3 by Namco. It's a fairly old game but you can customize a lot of parameters from individual bone size to eye width, depth, vertical/horizontal spacing, lash and brow styles, nose and lip shapes, sizes and positioning. There's also hair styles and skin hair color options as well. Obviously it's still limited to young heathly male/female characters but it shouldn't be hard to increase the size of the database to add more parameters like age. I'm pretty sure you could add a "create random character" button if you just want to make one really quick.

For anyone interested in trying it out the game is like $10 now. The game itself is pretty nice too similar to Virtua Tennis.

 

[TheWretched]

Thinking back to say Grand Theft Auto 4... there was a large handcrafted cityscape with basically no interacting within the houses. Most doors were simply just textures.

I mean, think of house interiors. There's NOT a lot of variety to them. There's a staircase and rooms left and right. Some windows, too. The contents of those rooms don't even have to make a lot of sense, either. Living room, kitchen, bathroom. There's not to that either.

A lot of people complained about GTA4 not having any interiors, besides the ones used in cutscenes and story missions. By generating rooms and interiors, this could've been avoided altogether. If it made sense, I do not know (I expect a resounding no here, to be honest), but you could generate this. The assets were all there, too.

I mean, you don't need a lot of info, either. Give the generator a bounding box of the house it has to generate... and maybe to metadata like location (rich or poor neighborhood), inhabitants (male, female, style) etc... the rest CAN be generated. A lot of CAD tools already can do this. Of course it will look less "fleshed out" than what an artist can do. But for a game like GTA4, you'd need several man years to do this, which is simply not the way to go, especially going further, where features like a fully fleshed out city will be asked by the consumers.

Thinking of say From Dirt. They generate vegetation on the fly. And it does look quite pleasing. I'd gather there's a lot of aristic work involved too, but it can generate these landscapes. And by offline prerendering such vegetational landscapes, there's a lot more you can do. The city scapes in Matrix 2/3 were generated like this from prefabs.

 

[Shifty Geezer]

I mean, you don't need a lot of info, either. Give the generator a bounding box of the house it has to generate... and maybe to metadata like location (rich or poor neighborhood), inhabitants (male, female, style) etc... the rest CAN be generated.

The important point for me is that, going forwards, this is the only way to do it. We're going to have to have automated aspects working with the artists assets if games are going to become more open and/or varied. Otherwise we essentially hit the limits of game design last gen, and are only adding more static stuff as RAM increases. Software hasn't been standing still all that time, nor will it, and just as there have been advances in other aspects to a game engine, procedural content will progress too.

 

[Arwin]

The important point for me is that, going forwards, this is the only way to do it. We're going to have to have automated aspects working with the artists assets if games are going to become more open and/or varied. Otherwise we essentially hit the limits of game design last gen, and are only adding more static stuff as RAM increases. Software hasn't been standing still all that time, nor will it, and just as there have been advances in other aspects to a game engine, procedural content will progress too.

Couldn't agree more.

 

[orangpelupa]

about that image http://media.tumblr.com/tumblr_lhcyji7UdC1qc9wc7.jpg
certain kind of japanese pc game and animation often use one "face template" for different characters.

this some example:

http://www.thebest4unow.com/anime-blog/wp-content/uploads/2011/01/air.cover1_.jpg
http://www.mangakoe.com/wp-content/uploads/2008/07/airtv_anime.jpg


maybe it can work good or not good also depends on the hole art direction of the game itself.

EDIT:
sorry actually it seems that xbox game.

 

[Colourless]

Our brains are designed to recognise faces. People can quite drastically change their appearance via changing clothes/hair style, but changing our faces not so easy. So if you see two people with the same face, but otherwise differing apperance our brain goes, its the same person (cause knowing who someone is kind of important in real life). Depending on the art style (such as anime), it may not be so much of an issue because its not realistic, but as developers strive for photo realism then its will cause more problems.

 

[Shifty Geezer]

That's all by-the-by though. There's no alternative, unless someone here can present one. How else can we get 100 different people models into GTA5 without procedural content, such as cut-and-paste components or algorithmic feature variation (which incidentally works very well in EA's sports games which shows it can be done, though of course that's a very different context to a city populace)?

Those decrying procedural content are basically just telling us to give up and accept things will never improve!

 

[Arwin]

Note also that in the examples below the expression on the face is different. With the faces being so primary, that makes a relatively big difference. You'd still think they were related though, like twins. (they don't look exactly the same either after all). But it is an important feature - if you give npcs a dynamic expression that makes them more unique too. You could achieve the same with posture, animation. These things start to matter at a distance that makes the face less clear.

 

[pc999]

I think that one of the problem for having good procedural art is that to do it you would need some kind of AI and we all know what AI is, unless it is for chess most most most of the time is a big fake or a big $&%&%%/.


I think you should talk and complain about the tools and their creators that keep adding thing just for better gfx but not simplifying how they are done or adding workflow improvements.


If in the last 10-15 years we had a major boom of movies, images and music produced it is because the tools got cheaper and easier to use. Today 400-500€ gives you everything you need to do a AAA work in some fields in 1/10 of the time. Actually if in somethings open source software arent up to par is because workflow and time saving features IMO.

Here are good examples of such tools in other multimedia forms
http://www.youtube.com/watch?v=8OPlD4N11vM
http://www.youtube.com/watch?v=vfkjHnsAsvg

You could do all (or almost all) of that in the older versions, but it would take at least ten times the time.


Then you could have more/better art and human made art.

We see the start of this in RAGE, that is a great looking game and the most talked features are tools for the devs and artist.

 

[patsu]

I think that one of the problem for having good procedural art is that to do it you would need some kind of AI and we all know what AI is, unless it is for chess most most most of the time is a big fake or a big $&%&%%/.

Not necessarily AI but perhaps high level decision or information.

e.g.,

Procedural is a broad term and might mean different things to different people. I don't know about Uncharted's animation blending, but even if it's simple keyframe interpolation I guess you could consider it to be procedural. This technique is so old/common that I don't think people consider it when talking about procedural stuff.

In Uncharted 2, they also implemented muscle emulation to prevent the blended animation from misbehaving. That muscle emulation system is essentially (one of) the high level logic to guide the blending.

I think Laa-Yosh brought up good points. However I think the word "random" appears too many times in his last post. A proper procedural system should not be "random" in nature. It should be guided or based on human, or other logic.

For the facial expression example, if it's too hard, then developers won't start with it yet. There should be other (many ?) low hanging fruits. I'd wait for technology to mature further. Perhaps crowd source the NPC face data from player base when PC/Mac/Pad cameras are advanced enough (high resolution and "3D") ?

 

[pc999]

Not necessarily AI but perhaps high level decision or information.

Whats the diference???



Anyway, even in the most simple examples you need to have smart decisions.

Lets say a "random" bedroom generator. For that to be good enough it will need to be smart enough to eg:

1- not place things in the player path;
2- not put a bed in front of a closet;
3- not put things in strange angles, like triangles between a bed and a table;
4- not put photos facing the wall;
5-...

That is just not to brake the experience, having you LOL hard at the game (best case scenario).

Probably you could make rules for everything but it would be a monster work, fallible and needing to deal with a innumerable number of variables.

Now lets think you want a specif atmosphere in a game:

In a horror game, you want many little clues and more clues per sale as you get near, like somethings that are "off", windows blocked, small warning with arrows, blood trails...This need quite a human understanding a visual communication skills.

In a game like bioshock, every room does have a certain personality, you would need skynet v4.5 or to re-program it to every level.

In a multi-player level not only it isn't as important, ass stuff can really brake the game, placing safe places(or stuff that looks that way) in the wrong places, giving misleading looks, getting in the player paths...

I think one could add lots of example for almost any kind of scene.


The only place I can see it being more useful is where our brain have harder time to register like trees, being quite complex, but in the same time it would be harder to register repetition or just small variations, so they can probably be artist made anyway.


IMO they really should take the example of guys like adobe and give tools to the dev/artists.

 

[patsu]

Whats the diference???

Depends on the sophistication and techniques used. e.g., it could be simplified physics instead of heuristics and inferences.