RSX Audio: Thanks DeanA

[Nerve-Damage]

Thanks goes to DeanA for clearing up the audio issue being presented at the Playstation Dev meeting in March. (http://www.beyond3d.com/forum/showpost.php?p=685088&postcount=33)

It's on that PS3 DevStation agenda because it *is* related to PS3's RSX. It's not discussing a pre-1999 sound system for the PC.. :roll:

I'd have thought this would have been pretty clear.. but these forums have a habit of dragging threads into 'odd' areas.. :)

Dean

Honestly, I don’t know why people believed the RSX audio being presented at the March Dev meeting wasn’t dealing with the RSX GPU.

The Cell is a great and powerful CPU, but too over rely on it (AI, Physics, Sound, Video decoding, ECT…) is begging for trouble IMO.

[VG Aficionado]

I'll say again that I believe Nvidia stated long ago their GPUs could be used as powerful sound processors, but I fail to see the need of sound processing on the RSX when PS3 has Cell.

[Mononofu]

I'll say again that I believe Nvidia stated long ago their GPUs could be used as powerful sound processors, but I fail to see the need of sound processing on the RSX when PS3 has Cell.

It does sound(oh no) a tad odd but I think abit more practical than trying to use your gpu for complex physics calculations. I mean if Cell could use that extra horsepower in more important areas why not?

[xbdestroya]

It does sound(oh no) a tad odd but I think abit more practical than trying to use your gpu for complex physics calculations. I mean if Cell could use that extra horsepower in more important areas why not?

I would say though that RSX having all it's processing power available for graphics is more important than Cell having all it's processing power available for physics, etc... It just doesnt make sense to divert any of RSX to audio should it result in a loss on the graphics side of things. Cell has 7 SPE's - they are perfect for sound. I personally don't see what is going on with 'RSX Audio,' but I imagine we will understand the situation better in a month or two.

[sif]

Maybe it is something to do with HDMI having soound and picture - so the sources both need to come from RSX?

[BlueTsunami]

Its almost as if Cell+RSX is one big special Virtual CPU that handles everything you throw at it (a sort of processor that is a CPU and a GPU all at once). The line defining what a CPUs job and a GPUs job is in this case seems to be blurring. I find that pretty cool

[Mononofu]

hmmm I don't know if this helps (can't insert links sorry)

This is from the Efficient 3D Audio Processing on the GPU paper (search on google)

I can't comment on a most of this but a few lines seem relevant.However, several shortcomings still prevent the use of GPUs for mainstream audio processing applications:

* Due to limitations in texture access modes and texture-size, long 1D textures cannot be indexed easily.
* Infinite impulse response (recursive) filtering cannot be implemented efficiently since past values are usually unavailable when rendering a given pixel in fragment programs. As suggested in [2], including persistent registers to accumulate results across fragments might solve this problem. Improved vertex processing available on the latest GPUs might offer another solution. This would also allow GPUs to be used for complex physically-based sound synthesis (e.g., modal synthesis [3]) in games or simulators and accelerate costly plug-ins for audio sequencers and multi-track editors.

BionicFX NVIDIA GPU Audio Effect Processor
meh

and SYMPHONY: Realtime Physically Based Sound Synthesis for Large Scale Environments

Was'nt someone on B3D talking about this? Instead of pre-recorded sounds play over and over again, having actual soundwaves travel and be morphed by physical objects and the environment?

My understanding of all this does'nt go that deep so I will stop here.

[Danji]

It's all very very weird to me. Something like 2 A/V outputs should not justify two sound processing units so that can't be the reason.

Maybe it's just there so the dev's can utilize the cell even more and not "waste" an SPU doing audio or perhaps it just that they wanted a more friendly audio DSP. In any case things will be a lot more clear shortly.

[xbdestroya]

See, what NVidia is aiming for I'm not putting down in the least - and in my desktop it might make a lot of sense. In fact I remember them discussing bringing 'SoundStorm' onto the GPU a little while back. But I just don't see how it would be a better use of resources than to use Cell.

Of course Sif might be correct and it just goes back to HDMI, but would that really warrant a whole presentation at Devstation?

[sousuke]

I'll say again that I believe Nvidia stated long ago their GPUs could be used as powerful sound processors, but I fail to see the need of sound processing on the RSX when PS3 has Cell.
I don't think the problem is the processing (cell will do sound processing anyway), but the output. The video and audio signal both have to go to the TV anyway, so why not combine audio and video in one chip (the GPU) instead of using two seperate chips.

[VG Aficionado]

I don't think the problem is the processing (cell will do sound processing anyway), but the output. The video and audio signal both have to go to the TV anyway, so why not combine audio and video in one chip (the GPU) instead of using two seperate chips.Because inside PS3 there are at least these two "separate chips" already to be used for their own purposes and you can't assume the sound always goes to the TV (I for one only use optical output).

[Lekko]

I remember back with all the technical demos when KK was talking about audio processing on the Cell. He was saying that sound would be more object oriented. (remember the leaf tornado demo?). So I think Sony was more planning on using the Cell to do their sound processing. Not saying that the RSX can't be used for audio though.

On another side, what about other chips? Isn't the PS2 EE supposed to be in there somewhere for backwards compatibility? I'm sure that is far more than enough for audio freeing the Cell and RSX for their respective tasks.

[section]

more shameless B3D quotes from Heavenly Sword team's "main man" Deano:



DeanoC


Originally Posted by Edge
I've touched on this before also, and it's not hard to see that a 3.2 GHz processor (the SPE) with 128 x 128-bit registers, and 256 KB of localized memory is overkill for just sound processing in a game. One quarter (one quarter of the processing, registers, and localized memory) of a SPE's total resources is plenty of power for sound processing, and the rest of the SPE could be used for something else.


Nope, we are allocating an entire SPU for sound.



Originally Posted by Edge
A typical game level would have no more than 5 to 20 MB of sound data, and the SPE would have no problems dealing with such an amount of data, and processing it.



Audio budget of a level is more like 300Mb+ of audio, excluding music.



DeanoC


Originally Posted by Edge
Cool! Is that because of a need, or because you guys are really pushing the envalope?



Bits? Like roughly 40 MBytes? If so, that higher than what I expected.

Bytes before compression...


so they really are pushing the envelope with their sound model and I'm not at all surprised they will "sacrifice" whole SPE only for the sound. As far as I've understood there are no audio helper chips on RSX, just audio out via HDMI and additional optical outputs.

[VG Aficionado]

On another side, what about other chips? Isn't the PS2 EE supposed to be in there somewhere for backwards compatibility? I'm sure that is far more than enough for audio freeing the Cell and RSX for their respective tasks.I wonder how they will enable backwards compatibility on PS3. I remember reading something about Sony hiring emulator programmers to create a software PS1 emulator for PS3. I wonder if they will use a PS2 emulator too, that should be cheaper than including an EE (+GS???) inside, although it could have more uses.

[RavenFox]

Well this is what Barbarian had to say on the subject. Same page as the link up top.
There is no actual audio work done on the RSX. It has some tiny audio component to pipe digital signals through the HDMI. All audio processing is done on the CELL - on one or more SPUs since they are really well suited for the job. This doesn't mean audio would tie up several SPUs for the entire time - a proper management system would allow other work to be done as soon as audio has finished its current workload.
Well some of you were right in the aspect of HDMI output. Looks like Cell is handling all the audio chores though. Im actually relieved to hear this.

[Nerve-Damage]

Well this is what Barbarian had to say on the subject. Same page as the link up top.

Well some of you were right in the aspect of HDMI output. Looks like Cell is handling all the audio chores though. Im actually relieved to hear this.

I have a hard time believing anything coming out this guy mouth!!

[xbdestroya]

Why what's wrong with Barbarian?

[RavenFox]

Huh? Whats up with Barbarian?

[rpgamer_2k5]

I'll say again that I believe Nvidia stated long ago their GPUs could be used as powerful sound processors, but I fail to see the need of sound processing on the RSX when PS3 has Cell. Yes, most GPUs that use high level shading languages could be used for audio purposes. In fact, audio programs that use the GPU for producing sound has been available for some time.

As I said before, I think the RSX will be a general-purpose GPU. It will be primarily designed for graphics, but also able to be used for audio and maybe even physics, neural networks (AI) and so on. It has been done before. :angelgrin

[Helios]

Honestly I never thought audio was that high on the list of calculation critical processes. To be frank I figured it was pretty much the least CPU heavy calculation in the entire game. Certainly 5.1 Dolby Surround might reqire more than a typical stereo output, but I dont see it biting into CPU time.

Maybe someone could clear that up.

[cpiasminc]

Honestly I never thought audio was that high on the list of calculation critical processes. To be frank I figured it was pretty much the least CPU heavy calculation in the entire game. Certainly 5.1 Dolby Surround might reqire more than a typical stereo output, but I dont see it biting into CPU time.
You're mostly right on that one. If people wanted to do some more extreme simulation type of things like simulating transmission, diffraction, reflection of sound, then you have a potential performance hit, but nobody does anything like that... yet...

Most any mixing and decode routines are faster done on the CPU than anything else. Modern sound cards, while they will do it all at very high quality, still can't outperform software mixing on the PC. That was even true on the Dreamcast as it were.

[Saibo]

You're mostly right on that one. If people wanted to do some more extreme simulation type of things like simulating transmission, diffraction, reflection of sound, then you have a potential performance hit, but nobody does anything like that... yet...

Most any mixing and decode routines are faster done on the CPU than anything else. Modern sound cards, while they will do it all at very high quality, still can't outperform software mixing on the PC. That was even true on the Dreamcast as it were.

Isnt 3D spatial sound synthesis as expensive or intensive as doing something like realtime global illumination? If so,if the RSX can do 3D spatial sound..wont that mean it could probably do some form of realtime GI?..... interesting times ahead.

[cpiasminc]

Isnt 3D spatial sound synthesis as expensive or intensive as doing something like realtime global illumination? If so,if the RSX can do 3D spatial sound..wont that mean it could probably do some form of realtime GI?..... interesting times ahead.
Pretty much, if you're talking something really "proper". In fact, a full exhaustive simulation is actually pretty similar in concept to GI, except for a few little details. For instance, transmission through a medium is frequency-dependent (lower frequencies move through more easily than higher frequencies).

In practice, things like transmission in a real-world simulation would be handled by setting the material properties of a wall such that its transmissive properties would be little more than parameters for a lowpass filter, and you just do the lowpass filter if the player is on the other side of the wall from a sound source. Which is a far smaller load than a full simulation. And that sort of signal processing type work is something SPEs will definitely do well... and I'm fairly sure that the SPEs have a few special instructions for just such a purpose.