Sweeney presentation on programming

[DudeMiester]

The biggest problem with the many domain specific languages model that has been suggested is the problem of communication within teams and with outsiders. Effectively, for every particular programmer and project you encounter you would have to learn a new language, because everyone inherintly thinks differently and thinking varies per project. This would be competely impractical in the real world, and I imagine it would lead to far more spagetti code, errors and other problems then we have today.

Instead I imagine a scalable language, where you have a very general and abstract framework to tie everything together, but when you get into those domain specific areas, you move to a lower level of coding. This is sort of like using ASM in C, C in C++ and C++ in C#, except that there would be better standards across the scale of languages. I think the reason C++ is so popular today is because it has this sort of scalabitiy, in that it can be very general or very specific as you need it. Although, I suppose you could also consider this a more structured case of the many domain specific languages model.

 

[AlgebraicRing]

The biggest problem with the many domain specific languages model that has been suggested is the problem of communication within teams and with outsiders. Effectively, for every particular programmer and project you encounter you would have to learn a new language, because everyone inherintly thinks differently and thinking varies per project. This would be competely impractical in the real world, and I imagine it would lead to far more spagetti code, errors and other problems then we have today.

http://www.martinfowler.com/articles/languageWorkbench.html -- taken from my new reading into domain specific languages.

I don't think domain specific languages are going to pose the problem you suggest they will. The reason I say this is because DSLs are not generic. The DSL is going to map straight into the language and ontology you already have for the domain. If you know the domain, you already have the basic ideas in your head and you're not going to have to learn a new programming language. Or at least what I am saying is true for standardized domains. Now if you are the creater of your own brand spanking new domain, you're right. But I'm thinking more about standardized domains that are talked about in the world of mathematics, chemistry, biology, engineering, etc. The concepts are pretty much ubiquitous and you would not be learning a totally new language. The less general and more specific it is, the easier it is to learn.

 

[compres]

I never heard of domain languages before, but from the sounds of it I am starting not to like it.

It's like the world today. We have many countries with different languages. Some countries are good at some stuff, othesr better at other stuff, but to comunicate(and trade services and stuff), you need to know each other's language. So you are basically left with many languages for achieving the same you would with a global language.

What's the point on that? Why would you like to speak french, spanish, english, mandarin and japaneese if you could probably do all the wortk in english?

OK, I know, enough with the analogy. From a software point of view I see no advantage what so ever. Haven't read enough about it though, so maybe I need to be enlightened. I suspect this matter has been debated quite a bit.

 

[Danalys]

it can translate between languages tho.

 

[MrWibble]

My problem with what Sweeny is saying, is that he's addressing how to express a concept in a better way, but none of this really deals with how to translate that expression into something the computer can actually execute efficiently.

His example of qsort in C++ and Haskell illustrates this - yes, the Haskell version is much much neater, but the compiler will either have to be orders of magnitude smarter than what we have right now, or it'll be slower than the C++ version by the time it's compiled - both versions will have to be translated into the same language to actually execute on a CPU...

Besides which, the C++ version is only complicated looking because it's already been written in a reasonably optimal form - I could write a less efficient C++ version that also looks neat.

The analogy of spoken languages is reasonable here - there have been attempts to design new languages that express things less ambiguously (than for example, English, which is really a horrible language in so many ways) - but AFAIK they've all failed to take off. I'm thinking that sometimes it's just better to adapt the language we all speak as the world changes, rather than try to get everyone in the whole world to learn something completely new. Sometimes we learn more than one language, other times we just splice bits of language together and grow our native tongues to encompass and express foreign ideas.

So perhaps there are elements of other languages that us C/C++ folks should learn and try to integrate into our "native" language, but I'm not sure we should adopt an entirely new way of writing code until someone can show me that it's actually a solution and not just a rephrasing of the problem.

Sweeny says he'll take a 10% productivity gain for a 10% performance hit. I'm not sure it's that simple an equation, and I'm not sure how far that scales either. Would he take a 100% productivity gain for a 100% performance hit? What does that even mean anyway?

If I could produce a game identical to how Unreal 2k7 will turn out, but do it all by next week using some uber-language, but it runs a 1 frame an hour - is there any point? Clearly there has to be limit to what performance is acceptable for a real-time application, and it's not going to be far away from what we have with C/C++...

I'd rather see more research go into finding ways to scalably parallelise traditional algorithms (or find replacements for those algorithms that will) rather than simply find a way of writing it down.

Of course he's got a flash car and a swanky house, whereas I live in a flat and catch the train every day, so he's obviously doing something right.

 

[darkblu]

Of course he's got a flash car and a swanky house, whereas I live in a flat and catch the train every day, so he's obviously doing something right.

you think so? one of the things that has always bothered me in the contemporary western mindset is how people are willing to measure various things in monetary terms. to try make this somewhat relevant to this thead, the logic behind 'developer x is commercially successful so his view on technology must be authoritative' somehow escapes me.

 

[Squeak]

Well, it only was "something", what that something was is yet to be clarified.

History is full of incidents where some person did "something" right, but was horribly wrong about anything else, but because of zeitgeist, luck or whatever, had huge success for a period of time...

 

[MrWibble]

you think so? one of the things that has always bothered me in the contemporary western mindset is how people are willing to measure various things in monetary terms. to try make this somewhat relevant to this thead, the logic behind 'developer x is commercially successful so his view on technology must be authoritative' somehow escapes me.

All I'm saying here is that his technical knowledge has gotten him pretty far, so he must be doing *something* right. Maybe he just got lucky, but I think there's more to it than that.

There are plenty of people in this industry driving flash cars who I wouldn't trust to fetch my coffee, let alone try to encourage fundamental changes in who we write code - but where success has come from someone proving they actually know their stuff (i.e, over a period of years and several products, they've built up a business around their skills which has yielded big piles of cash), I'll not dismiss their ideas out of hand, even if my initial reaction is "WTF?"

But I don't consider him *authoritative* and neither would I say that about Carmack or anyone else. Success does not imply authority, but it can lend credibility.

 

[Robert.L]

Well, it depends on how much work the main thread is doing versus the heavyweight rendering thread. It appears that the ideal hardware model for the unreal3 engine would be 1 or 2 heavyweight processors with a number (4-6) lesser processors hanging around to handle the helper threads. This arguably looks more like the Cell configuration than the Xenon. Still, there may be advantages either way. Cell should have a slightly beefier PPE than any of the three cores in Xenon, but the SPEs aren't as flexible or powerful. I assume Xenon would run the Main thread on one processor, the Rendering thread on another, and the rest of the helper threads on the remaining processor and the hyperthreading "processors".

On Cell, you'd have to figure out how you are going to split the rendering thread and the main thread. Which one runs on the PPE, which on one runs on one of the SPEs? The helper threads I'd assume would run on the SPEs.


Cell in PS3 has one of the SPEs disabled, so you have the PPE+7SPEs which is probably how they got that number. I don't remember if the PPE has any kind of hyperthreading like functionality, but it appears epic isn't counting it if it does...

Nite_Hawk

Don't know about flexibility of spe’s, most likely that will depend on the programmer but spe’s are powerful little bastards their theoretical fp and fixed/integer performance is impressive.

 

[AlgebraicRing]

My problem with what Sweeny is saying, is that he's addressing how to express a concept in a better way, but none of this really deals with how to translate that expression into something the computer can actually execute efficiently.

His example of qsort in C++ and Haskell illustrates this - yes, the Haskell version is much much neater, but the compiler will either have to be orders of magnitude smarter than what we have right now, or it'll be slower than the C++ version by the time it's compiled - both versions will have to be translated into the same language to actually execute on a CPU...

Besides which, the C++ version is only complicated looking because it's already been written in a reasonably optimal form - I could write a less efficient C++ version that also looks neat.

Here is the difference between a procedural language and a good declarative or functional language. In order to express an idea in a procedural language, like a sorted list, you are forced to write the implementation of that idea and then your implementation is used to enforce the idea. In a functional or declaritve language you should only be saying enough to express the idea and it is the job the compiler to find an optimal solution. For cannonical patterns like sorted lists, the optimal implementation for it, given the data representation, should already be baked into the brains of the compiler or interpreter. The ideal expression of a sorted list is as follows:

S is a list of some type A
<= is a comparison on elements of type A
S_x is the xth item in S
The following is the optimal expression of a sorted list.
Sorted(S) if and only if S_1 <= S_2 <= ... <= S_size(S)

The expression of a sorted list at the most abstract level is so uniform and frequently used that the actual algorithms for sorting should be baked into the compiler. All the compiler has to do now is 1) notice that what is being expressed as a constraint is that the list needs to be sorted and 2) perform the most optimal sort for the current data-representation of the list.

* If you are writing procedural code, you are telling the compiler how to solve the problem.
* If you are writing declarative code, you are telling the compiler what you would like done, but not how to do it.

Which method is better? In the first case you can solve your problem however you like using as optimal a solution as you like. For now it is necessary to tailor algorithms and their implementations to the quirks of embedded systems. This gives fast code (provided you're good at what you do. On the other hand you are taking your time to write solutions to problems which have been solved 1000s of times over by 1000s of different people across the globe. You're re-inventing the wheel and often it is not even to find the solution on a quirky hardware.

In the second case, you solve the problem once, formulate a concise expression for it and add the know-how of the solution to the compiler so that other people can begin to use your solution. What gets optimised then is the compiler and not the expression of the idea within the language. In declarative languages you need only invoke the idea. Some ideas are procedural in nature and their invocation will look procedural. Ideas which are not procedural will not look procedural. This gives massive amounts of freedom to the compiler to find an optimal solution. When you are writing a declarative problem in a procedural language you are being FORCED to pick 1 and only 1 method for solving the declarative problem in a procedural way.

Sorting is a declarative problem. The compiler should know that what is being expressed is the desire that the data be sorted and it is the compiler which should figure out the best way to do that given the data representation of the list. In declarative languages the actual data representation is hidden from the programmer. The programmer simply expresses his ideas as succinctly as possible.

Right now I'm of the mind that procedural languages should only be used in writing compilers and declarative languages should be used to express ideas about data manipulation.

The analogy of spoken languages is reasonable here - there have been attempts to design new languages that express things less ambiguously (than for example, English, which is really a horrible language in so many ways) - but AFAIK they've all failed to take off. I'm thinking that sometimes it's just better to adapt the language we all speak as the world changes, rather than try to get everyone in the whole world to learn something completely new. Sometimes we learn more than one language, other times we just splice bits of language together and grow our native tongues to encompass and express foreign ideas.

You're joking right? There is no such language as English. It doesn't even exist at an abstract level. We're all speaking with our own dialects and we can communicate in as much as they overlap. Our individual languages overlap most when we are talking about very clear domains and patterns of information. Medical doctors have their own domain specific language that they use. Business salesmen have their own domain specific language. Each sub-culture has its own domain specific language. The domain might use lots of words that you speak normally and the meanings of those words might not stray very far from what you would expect. But the fact that the words are used within a domain means there is a force which is anchoring the meaning of the words. When the words are spoken within the domain, it is the meaning of the domain that is invoked and not the meaning of your ordinary vocabulary. We intercommunicate best with each other when there is a clear domain to anchor the meaning of the words being used. A society will fracture if there are no domains to anchor language and meaning.

So perhaps there are elements of other languages that us C/C++ folks should learn and try to integrate into our "native" language, but I'm not sure we should adopt an entirely new way of writing code until someone can show me that it's actually a solution and not just a rephrasing of the problem.

From a theoretic standpoint, why code a C compiler by hand for each new processor? What you need is a meta-framework where the theory of finite computation is known and the definition of the processor will co-mingle with the definition of C and its known algorithms such that a compiler for C on that processor can be generated automatically. The meta-framework is a compiler compiler, and it saves a tremendous amount of recoding things by hand.

The cutting edge of technology is proof theory and automated proving systems. How long before these systems can find optimal implementations we've never thought about? 20 years? In 20 years you should be writing more declarative code than procedural code. The study of linguistics combined with the meta-theory of computation I think is going to revolutionize programming language theory. I'm hoping it produces some very nice tools in the future which will cut out procedural programming and will allow people to write declarative solutions behind which a compiler will break the problem down and farm sub-problems out to be solved by specialized problem solving engines, which they themselves might be multi-processor spanning.

 

[Frank]

I would like to add a simple observation to the parallelism problem: languages that have a fixed set of answers up front do much better than languages that start out with determining the work that has to be done.

For example, when you do embedded or pixel processing, you start out with a fixed set of ports or pixels that each have to receive a new value. Those are all "embarrasingly parallel". But when you write a program in the classical sense, you start with some conditions, that result after processing in changing an unspecified amount of unspecified values or objects, with dependencies.

While it might look extremely inefficient on the outset to even process all the ports and pixels that won't change value, it isn't. Because you can essentially just add more hardware to solve it, while you cannot increase the performance of a single thread infinitely. So, if we take games as an example, changing the paradigm should be enough at first. Most of the general computing resources of the new consoles aren't used at all, just because they start off by calculating things like which objects are visible and/or interact, and have to be processed, which requires a single thread.

A simple way to leverage all that extra power is to take the brute force approach: make a list of all the objects in the scene, and just calculate all of them, using a fast-out if they turn out not to need it. And when you use that approach, you can mask the latency and interdependence by keeping lists of things that still have to be processed, and just add things that have interdependencies to the end of the list. And use message parsing or pending property changes (parameters) to resolve locking conflicts.

While that sounds woefully inefficent, it would be a simple way to use all the processing power available. And I think it would perform much better than you would expect, mostly because you don't have to keep or check states of other objects.

The only thing you have to figure out to make the above really work is a better (read: more cache/local store friendly) way of storing your object data. Which can absolutely be done with current programming languages. But perhaps not with the current object model.

In short: start thinking at the other end of the pipeline.

 

[Frank]

Btw, there already is a very good mechanism to tackle the interdependencies and locking using transactions: just look at how any SQL server does that. It might increase the latency, but we're talking many thousands of objects, so who cares.

 

[ERP]

FWIW I don't think we'll be seeing any revolutions in programming any time soon.

Garbage collection will be the norm or at least an available option, and we may have more ways to constrain data, but we'll still be using C/C++ to pound out games.

They were talking about the coming functional language revolution when I was at University 18 years ago. Miranda was the big thing back then they'd just built a working spread sheet that ran at decent speed using the language, and there was talk of processors tailored to the requirements of functional languages and lazy evaluation.

Here we are today having the same converstions 18 years later.

And functional languages don't solve the thinking about the implementation issue. I spend as much time in a piece of ML code that I need to be efficient as I do in a piece of C++ code. I understand what the compiler can and can't optimize and I tailor my code to it just the same, I'll add extra variables to a recursive function so a solution can be expressed in terms of tail recursion. The compiler identifying what your trying to do and selecting the best implementation is really no different than calling into a library and the latter is much simpler.

The problem is that the best language doesn't win Objective C is a much better designed language than C++, but that does't make it more poular. Language selection is a very pragmatic process. Languages like C# and Java gained traction because they look a lot like C++, they have excellent tools and they make programming easier.

The problems we have now are really two fold, one is an overall architectural issue, how do we build large parallel systems, at some level languages are irrelevant in this discussion, and how do we implement the pieces in this new architecture in this area the right language can make things a lot easier.

C++ has a lot of bad assumptions for parallelism, not the least of which is that everything is by default mutable. unfortunately I don't see this being enough to kill the language.

As to massive parallelism, and doing everything in parallel with early outs, prallelism is only a win when the communication time is much smaller than the computation cost, right now that means we have to do a significant amount of work per unit to make it worthwhile. GPU's have the advantage of being able to sclae each part of their pipe as they become the bottlenecks, internal communication innefficient, build a ring bus etc.

 

[Frank]

As to massive parallelism, and doing everything in parallel with early outs, prallelism is only a win when the communication time is much smaller than the computation cost, right now that means we have to do a significant amount of work per unit to make it worthwhile. GPU's have the advantage of being able to sclae each part of their pipe as they become the bottlenecks, internal communication innefficient, build a ring bus etc.

That's not lateral thinking; it's stating problems. Do we want a better way, that puts all the processing power available to (good) use?

And for garbage collection as well as parallelism: scope is everyting. That's why a garbage collector is a bad fit for a JIT compiler, as well as for C++, although for different reasons. For a JIT compiler, because it cannot determine the scope (so, while it will prevent memory leaks, it will at all times use much more memory than needed), and for C++ because of the mutability (so it won't know for sure if some pointer won't be cast into a type that might represent a variable that is cast to that same type as well).

Functions and streams are easy to thread, as long as they use no globally available data. Objects on the other hand, are designed to keep their own instance consistent, not the global access to the heap. Wich is just the wrong way around.

While a model with independent objects doesn't change the requirements of things like collision detection to have a spatial representation of your objects (probably the main problem in any case) and the need to look them up, it does change the amount of times you would do that. Because, collision detection is just a possible interaction to the object, alike to seeing an enemy, resting on a surface while subject to gravity, or triggering any other action.

You might be able to make more intelligent decisions, because you don't have to take all cases into account. Like, an object that has no long distance interaction doesn't have to check for those. While with the current model, you check if such an interaction is possible, and then start looking for objects that have that possibility.

 

[Frank]

Btw, for collision detection, you could get rid of the requirement to iterate the possible interactions to closure when using the above method. Just add the possible objects to the end of the list.

 

[Quaro]

Is there any chance whatsoever of getting the audio to this presentation? It would be fantastic.

 

[Entropy]

Many C/C++ programmers, in the console world, do not use functions such as malloc/new or free/delete (which in turn cause the OS to intervene each time) much in the program: static allocation, one BIG malloc() at the beginning and then you use custom memory management functions that are faster

Old fart input - we used to do the same in FORTRAN - declare one huge COMMON block, and then handle memory from there. Allowed implementing your own virtual memory schemes and other useful stuff back in the day.

you think so? one of the things that has always bothered me in the contemporary western mindset is how people are willing to measure various things in monetary terms. to try make this somewhat relevant to this thead, the logic behind 'developer x is commercially successful so his view on technology must be authoritative' somehow escapes me.

I believe it comes from the American "You can't argue with success".
Repetition seems to be made this into a more literal truth than you'd think, and it spills over into the weight attached to whatever is said by those regarded as successful, be it Bill Gates or Madonna.

Interesting discussion BTW. Get on with it. Interlude over.

 

[DemoCoder]

And for garbage collection as well as parallelism: scope is everyting. That's why a garbage collector is a bad fit for a JIT compiler, as well as for C++, although for different reasons. For a JIT compiler, because it cannot determine the scope (so, while it will prevent memory leaks, it will at all times use much more memory than needed)

The JIT aspect doesn't really have anything to do with whether or not scope is inferrable by the compiler. Some problems are statically solvable, others are dynamically solvable. Both static compilers and dynamic compilers can perform escape analysis to determine scope in a surprising large number of cases (just like loop bounds checking is eliminated in a lot of cases) Both the newest IBM JIT and Sun's JDK 1.6 Mustang do this, as well as many older VMs like Smalltalk.

For example, if escape analysis determines local scope, than "new"ed heap memory can actually be allocated on the stack and freed upon stack frame deactivation. As well as any thread sychronizations/locks/monitors/etc can be eliminated.

Moreover, the amount of memory used is tunable, and is not much different than the techniques game programmers use to build custom allocators, where a gob of memory is allocated up front. (my claim about C programmers has nothing to do with game programmers, but the flavor of anti-GC hysteria I have see in USENET from C/C++ programmers. Yes, elite coders will build their own allocators, override operator new() etc, but beyond that, C/C++ coders were claiming that explicit new/free/destructor calls were more efficient, and it just isn't the case)

The problem with the functional approach is this. Idealists say you don't have to worry about how the imperative implementation is carried out, so you are more productive, since you write algorithms at the specification level, and trust the compiler to do the right thing.


Reality Check Time: Real Haskell/ML production programmers have to learn the intricate details of how their compilers work, and how to "tweak" their code so that they trigger the appropriate constructs to be optimized probably. Make a mistake in Haskell and you can lose performance by a factor of *100*, I kid you not. Alot of it is to avoid boxed primitives, explosive memory buildup from lazy-evaluation, and getting recursion optimized.

Procedures like:

fib 0 = 1
fib 1 = 1
fib n = fib(n-1) + (f-b2)

Looks clean. They look great. I love the elegance. But in reality, this runs very slow. The result is, the code must be rewritten in tail-recursive form with an accumulator. Same goes from the reliance on continuation passing style everywhere. I think it hurts elegance, but improves performance.

 

[DudeMiester]

This is the problem with that sort of language, you have to guess, experiment, etc... until you figure out what works and what doesn't. It would be easier to know up front exactly what the compiler does and doesn't do, and how exactly you need to take advantage of this. Procedural languages do this better imho, because they are more explicit about execution.

At the same time, you could have a compiler that is plug-in based, allowing you to view explicitly what optimisations and tricks it can do. If you need something it doesn't have or want a hand-optimised low-level routine, just code another plugin. The compiler would scan through what's available and apply the best options.

However, what do you really have here with these plug-in compilers, lazy-evaluation and DSLs? It's really all just added layers of abstraction, which is something procedural languages can already do, even if it may not be maximally efficent. For example, I read that article on DSLs that was linked, and it seems to me that DSLs are just different views in the model-view-controller principal, which itself is basically just a long way of saying "abstraction". In MVC, you abstract the one interface to a more intuitive/practical one, like going from a database to a GUI, and this is exactly what DSLs are supposed to do. I think the real question that needs answering here is, what is the optimal way to abstract something?

Personally, I don't think that there is any one way because as DSL proponents say, it's a case by case basis, and this means that sometimes DSLs, functional languages and all that jazz arn't the right way. All I do know though, is that we absolutely need some framework to tie this all together efficently. Unlike some, I don't think it's god-like compilers that we need, that kind of idealism is just absurd. Rather, I think procedural languages are the only ones that can fill this role. They have scalability down to the lowest levels (ASM), which dynamic languages inherintly can't do. At the same time, they are perfectly abstractable, even if the syntax may not ideal. This is the only configuration that can yeild the generality necessary to be the glue that sticks all these other "expressions", like DSLs or functional languages, together.

Now I suppose this would also segment the programming feild even further from the current state of ASM vs OOP. Probably into something like the ASM guys that provide the bedrock/performance, the OOP guys that provide the glue/management, and the DSL guys that provide the configuration/content. Naturally, when performance is important, code will move down the chain into lower, more optimised, levels. Overall, imho things will look same as today, except for the layering of "abstract"-level or "expression"-level languages onto todays popular languages.

 

[DemoCoder]

A pluggable compiler won't help. Optimizations interact in complex ways. Let's say the compiler has two optimization plugins: Opt1 and Opt2, and you fully understand how Opt1 and Opt2 work and what code patterns trigger them. In general, you won't be able to predict Opt1 + Opt2 turned on together on your existing code base.


Also, DSL's don't have anything to do with model-view-controller unless you interpret MVC so liberally as to include almost anything that includes data, a "renderer", and a handler. For example, regular expressions, SQL, XPath, YACC, HLSL, are domain specific languages.