Musical tutor game possibilities* Spinoff

[Arwin]

I actually contacted Sony about that, modifying the SingStar engine to work with continuous pitch instruments like violins, where you need to learn muscle memory and an ear for the right pitch. Didn't get any reply though

I guess they don't think the market is there for training tools that are more complex than games. People won't be buying ViolinStar for the fun of it, as it were, because progress will be very slow. A real guitar would be better as placement of fingers has a lot of leniency thanks to the frets, and it's easy to get a sound from simple chords. Heck, many top guitar anthems use only three chords! Matching timing up with finger placement would be much better. Cost of the instrument would be high though! That said, if you had perhaps an optical system monitoring finger placement, and a mic to pick up rhythm on the strumming, you wouldn't need to do anything complex to monitor pitch accuracy unlike SingStar. That'd make the required peripheral...well, PSEye seems ideal. As long as you already have a guitar. Otherwise, again, it'll be expensive. Perhaps not much moreso than GH though. Entry level guitar prices are about the same price as a game here in the UK.

I symphatise, as I play several instruments. In theory you can use Singstar, because it really only does listen to pitch already, and it doesn't mind which octave you sing in either. So in theory, if you set it to the highest difficulty level, you can use it at least somewhat for this purpose as it is. Maybe it'll happen on PC first, and someone takes it from there.

Optical would be difficult because your hand blocks out your finger placement a lot of the time. The old midi methods would be a decent way to deal with this. For instruments that don't support midi, you have some intermediate software that can convert stuff to midi, but there are also midi-guitars. That kind of thing would be a good entry into the series. I don't see it happen any time soon, but it could be very cool.

I think it should be pretty feasible to use an electrical guitar and unamplified have the Cell recognise the individual strings you strummed and such. It'd be more difficult for violins, as you'd need a decent microphone.

 

[Shifty Geezer]

I think it should be pretty feasible to use an electrical guitar and unamplified have the Cell recognise the individual strings you strummed and such. It'd be more difficult for violins, as you'd need a decent microphone.

Or an electric violin . I think frequency detection is harder on some instruments due to harmonics, especially with chords. That's not a problem with voices! I wouldn't expect pitch recognition via audio of a guitar to be pretty rough. A quick Google shows there's a couple of software tools out at the moment at least, but the one with demos showed even with simple audio it was unreliable, and the other version had an editor for correcting errors. If it's anything like OCR that even on clean Times New Roman print can make mistakes (and chords should be a lot harder to distinguish than that) than I doubt it'd be reliable enough for a computer tutor on polyphonic audio.

 

[Arwin]

Well, US residents seem to get access to at least one existing game that actually helps you improve your singing, and it's happening on PS2 even before the PS3 release.

Remember, Singstar does actually detect your pitch and volume, and displays that realtime. You try to match the on screen bars with your voice and get points. There are different difficulty levels which basically have more or less strict matching windows. On Hard, you have to be singing pretty much on pitch. Because it is realtime, you can adjust your pitch immediately if you notice you're off and not scoring, which is great.

Singstar is coming to the US immediately in a neat PS2 white bundle too (4 november I think):

http://blog.us.playstation.com/2007/10/26/singstar-playstation-2-packs-get-‘em-while-they’re-hot/

 

[liolio]

given the processing power of the cell it would be super interesting to be able to connect a AD box, and run some amp emulator on the ps3 think about line6 products or Amplitube etc.

 

[Arwin]

There's a tonne of possibilities in the sound department with Cell. Nothing stopping an enterprising coder either - you could easily (well ...) set up Linux op the PS3 with the Cell devkit and start working on that right now I think.

 

[Shifty Geezer]

After the original discussion of pitch recognition, I did a bit of a search on find a few examples, including TS-AudioToMidi. I only had a brief look, but it seemed okay with translating the electric violin, and has several different detection modes. However demo MIDI files show it's still fraught with problems, and it'd have to be 100% accurate to avoid frustration. Still, if you were to work with a known instrument with a direct in, you may be able to get some solutions working. There was also a recent interview with a chap from Harmonix who was talking about making music games capable of teaching real instruments rather than just simulating them on a simple level, though I can't find it.

As for Amplitube, it already runs realtime with not much effort on my Athlon 2600. The real place to push Cell is in synthesis where it could handle awesomely complex modelling systems. Development will likely be hampered by hardware adoption though. Who's going to invest in super synthesis on Cell when your market is a few PS3 owning geeks?! Either it'll be created as a 'game', perhaps something like Cakewalk's Music Studio for PS3 - you've got an HDD so why not? - or more likely won't happen unless someone experiments via PS3 Linux. It's an area I'd like to have a bash at, but there's no chance at the moment. If I have to create a whole graphics drawing and UI subsystem before I can start on the fun stuff of audio synthesis, I'll be fed up with development before really starting! If you could get it do awesome things and very well, there might be a chance of packaging it up as a synth to sell musicians, but it'd need linkage to a PC and be controllable via CuBase, requiring more effort than a guy at home on his own can manage.

 

[Arwin]

If I have to create a whole graphics drawing and UI subsystem before I can start on the fun stuff of audio synthesis, I'll be fed up with development before really starting! If you could get it do awesome things and very well, there might be a chance of packaging it up as a synth to sell musicians, but it'd need linkage to a PC and be controllable via CuBase, requiring more effort than a guy at home on his own can manage.

The graphics drawing and UI subsystem can mostly be done using avialable Linux libraries though I suppose?

I do have most of the equipment needed for testing and developing this kind of thing at home myself (decent grade mike, instruments, mixing table, etc.). As soon as I get my 2nd PS3 I'm in ...

Sing Star finally has fixed and official release date by the way for the 7th of December. Awesomeness! Just in time to practice some Christmas Carols.