Massively Parallel Distributed Computing via Game Consoles?

[koldfuzion]

Hey all, this is my first post, although I've been reading these boards for several months. Basically, I have a simple, perhaps dumb question.

First, the basis of my question is this: based on M$ specs, Xbox 360 claims to have 1 teraflop (per specs on Tom's hardware). Assuming that is correct, if you have 10 million units, and you could use 1/4 of that idle processing power, you would have 250 million teraflops :shock: of processing power waiting around to be used.

I assume Sony will come in at at least 1 teraflop now that MS has claimed to hit the number. (I'm not sure if M$ is double counting GPUs and CPUs and such, but let's assume for a moment that the number is legit, disregarding whatever mental gymnastics M$ is using to come up with that number). If you then assume that number is inflated by a factor of four, you still have 250 gigaflops of capacity per unit. So if that is true, assuming ten million units "available" using my 25% guesstimate, idle capacity would still equate to a whopping 62.5 million teraflops. :shock: Obviously this also assumes 10 million broadband connections for each unit, and that some of this capacity could be tapped into, a big assumption.

Anyhoo, the latest list of supercomputers I saw had Blue Gene at number 1 in 2004 and it clocked in at 70.72 teraflops. Is there any way that distributed computing networks, like Climateprediction.net or SETI could tap into that idle capacity? :idea: Or is it just impossible in that there is no OS in these systems that could handle a distributed computing platform? I.e., is there no way to install software that can use the idle capacity since this thing isn't running Windows or whatever?

In a similar vein, of more interest to me as a gamer, my understanding is that the whole design of Cell is centered around being able to tap into the power of other cells through a network of Cells to ramp up its power using a parallel/distributed computing model. Is this possible/and or likely that each PS3 could tap into other PS3s over the internet to enhance and multiply the performance of each console? Or is that just a fantasy? :roll: Anyway, I'm no computer engineer, and I'm just doing some back of the napkin math here, so I'd appreciate it if someone could enlighten me. But strikes me that these consoles would leave a hell of a lot of cheap raw power just sitting around waiting to be tapped. No :?:

Off topic

M$ completely uninformative Mtv special sucked. The ourcolony commercial was at least somewhat helpful. The Tom's Hardware review is really helpful. It's posted elsewhere, I'd give credit for the link but if I hit the back button, I'll lose my post...

Resume topic.

Thanks anyone for your input.

[xbdestroya]

Well well - first of all, welcome to the forum, good first post. 8)


Hey all, this is my first post, although I've been reading these boards for several months. Basically, I have a simple, perhaps dumb question.

First, the basis of my question is this: based on M$ specs, Xbox 360 claims to have 1 teraflop (per specs on Tom's hardware). Assuming that is correct, if you have 10 million units, and you could use 1/4 of that idle processing power, you would have 250 million teraflops :shock: of processing power waiting around to be used.

MS' terraflop numbers are meaningless marketing numbers - The tri-core CPU itself will have a little over 100 GFLops of power, and as for the rest it's all coming from stuff (like the GPU) that couldn't possibly ever contribute to any sort of distributed computing tasks. To understand what framework Microsoft is working from when they give their FLop-ratings, go check out the flop ratings for the original XBox, and you'll now immediately how to decipher the new numbers.


I assume Sony will come in at at least 1 teraflop now that MS has claimed to hit the number. (I'm not sure if M$ is double counting GPUs and CPUs and such, but let's assume for a moment that the number is legit, disregarding whatever mental gymnastics M$ is using to come up with that number). If you then assume that number is inflated by a factor of four, you still have 250 gigaflops of capacity per unit. So if that is true, assuming ten million units "available" using my 25% guesstimate, idle capacity would still equate to a whopping 62.5 million teraflops. :shock: Obviously this also assumes 10 million broadband connections for each unit, and that some of this capacity could be tapped into, a big assumption.

Anyhoo, the latest list of supercomputers I saw had Blue Gene at number 1 in 2004 and it clocked in at 70.72 teraflops. Is there any way that distributed computing networks, like Climateprediction.net or SETI could tap into that idle capacity? :idea: Or is it just impossible in that there is no OS in these systems that could handle a distributed computing platform? I.e., is there no way to install software that can use the idle capacity since this thing isn't running Windows or whatever?


This is a good question, but I have to doubt that it would occur, as most users would be hesitant to farm their console out, and most companies would be scared of trying to convey the notion of even being able to to the consumer. Still, I wouldn't doubt Sony might indeed have some sort of 'grand network' scheme brewing; it's just I personally can't conceive of what it would be.


In a similar vein, of more interest to me as a gamer, my understanding is that the whole design of Cell is centered around being able to tap into the power of other cells through a network of Cells to ramp up its power using a parallel/distributed computing model. Is this possible/and or likely that each PS3 could tap into other PS3s over the internet to enhance and multiply the performance of each console? Or is that just a fantasy? :roll: Anyway, I'm no computer engineer, and I'm just doing some back of the napkin math here, so I'd appreciate it if someone could enlighten me. But strikes me that these consoles would leave a hell of a lot of cheap raw power just sitting around waiting to be tapped. No :?:



My answer above best addresses this, and specifically as it applies to gaming, I just don't see how it would be possible, as network latencies would essentially render any available Cells on the network meaningless for real-time 'need it now' power.


Off topic

M$ completely uninformative Mtv special sucked. The ourcolony commercial was at least somewhat helpful. The Tom's Hardware review is really helpful. It's posted elsewhere, I'd give credit for the link but if I hit the back button, I'll lose my post...

Resume topic.

Thanks anyone for your input.

Yeah, the MTV special was not the best.

[koldfuzion]

Thanks! Oh well, seems to me that leaves a lot of wasted firepower lying around. But sounds like the logistics of tying everything together in a meaningful way that improves performance are just too complicated. I figured as much. Anyway, appreciate the feedback, I think that answers my questions. :D

And what a shock to hear that M$ numbers are total BS. :wink:

Danke

[koldfuzion]

per another post from Mordcaii:
By the way... in an interview with Masayuki Chatani (SCEI CTO) http://www.beyond3d.com/forum/viewtopic.php?t=23162, he states that the Getaway demo was running purely on the CELL, with the image created by CELL directly output to the frame buffer... THAT'S CRAZY!!!
_________________
"True knowledge exists in knowing that you know nothing." -Socrates

From the article:

It has no router function. It's supposed to be a switching hub internally, but I don't know about the meaning of In/Out frankly so will answer about it at the next opportunity. Of course you can connect PS3s together. After it's on sale, some will make a supercomputer by connecting many PS3s. Apparently Sony Picture Entertainment is considering to use PS3 in a rendering farm for movies.

Does this change anyone's view at all?

[Mordecaii]

From what I've read in other interviews, the new Eyetoy is supposed to occupy one of the ethernet ports (since the new cam is IP based so it can broadcast around the world without your PS3 being there). The other port I believe is just to continue the connection along... It was stated that the PS3 was not meant to be the end of the network but to simply be a part of it... this would suggest that you can hook up another device to the PS3 via the network port and it should be able to access the net.

[Coded-Dude]

From what I've read in other interviews, the new Eyetoy is supposed to occupy one of the ethernet ports (since the new cam is IP based so it can broadcast around the world without your PS3 being there). The other port I believe is just to continue the connection along... It was stated that the PS3 was not meant to be the end of the network but to simply be a part of it... this would suggest that you can hook up another device to the PS3 via the network port and it should be able to access the net.

yes 3 supposed Gig NIC's(1 IN / 2 OUT)

which means you can hook up two more devices or a hub or a switch and make several more LAN's

But I doubt the eyetoy will have direct ethernet port acess. The PS3 will be its HUB to the internet
I think it will be bluetooth or USB and utilize the PS3's network, but I am guessing just as you are.

[DofD]

Well, no and yes. In a render farm setting where you have several hundred PS3's connected via their gigabit ports on a dedicated network, they could work quite well. In consumer environment over even broadband, anything more than a unit of work transaction system like SETI implemented would work. Mainly due to transfer speeds.

When implementing a massively parallel distributed computing environments the communications speeds between nodes and the reliability of nodes becomes very important.

With today's consumer technology, it's not practicle to implement a cooperative computing network to assist in node performance due to the limitations of communication speeds. The types of problems a games console has to solve need to be done almost instaneously. Whereas the problems perfomed on the supercomputers you mentioned can take from seconds to weeks to perform.

It's a nice idea though. If the communication speeds between PS3's were much (and I mean A LOT) faster it could be possible.


Cheers,

DofD

[Coded-Dude]

we are very closee to 10GB networks. Well, technically they already work, but are not standardized or public.

[the legendary ice man]

There are 1TB switches or routers - I forget which...

Anyway.

I'm sure that going back something existed like this as a built-in part of CELL technology. Being able to share idle power.

Although I believe it was broadband based and worked globally - not 100% sure on it though.

It sounds like something the CELL would be able to do in a next version if not this since it's supposed to be built on something to do with broadband transfer and engines... - correct me if I am talking crap ;)

[Coded-Dude]

No, thats virtually impossible(with current technology), I believe the fastet yield we have encountered is 98GB/s throughput.
But like I said, 10GB NIC's should start showing up soon :wink:

[the legendary ice man]

Researchers working under the Japanese Femtosecond Technology Research Association have developed an optical switch capable of processing one terabit of data per second.

A research team at Japan-based Mitsubishi Electric has developed a III-V optical device suitable for 1 Tb/s data transmission, according to the Japanese newspaper, Nihon Keizai Shimbun.

november 15, 2004 - Source (http://www.compoundsemiconductor.net/articles/news/8/11/13/1)

I rarely, if ever, lie - although I am human and do get muddled and facts jumbled.

[Brandon]

Researchers working under the Japanese Femtosecond Technology Research Association have developed an optical switch capable of processing one terabit of data per second.

A research team at Japan-based Mitsubishi Electric has developed a III-V optical device suitable for 1 Tb/s data transmission, according to the Japanese newspaper, Nihon Keizai Shimbun.

november 15, 2004 - Source (http://www.compoundsemiconductor.net/articles/news/8/11/13/1)

I rarely, if ever, lie - although I am human and do get muddled and facts jumbled.
That's quite insane, really.

[Coded-Dude]

Well, I stand corrected.....:oops: btw - I was not necessarily calling you a liar(just found it a little hard to believe - thats all)
Thanks for the source!
I wonder what kind of success they have had with it?
You'd think something like that would sell like Chicken McNuggets!

I support the networking division of a large company and we are currently pushing around 98GB/s with much success.
Labs are awesome to work for(the latest and greatest technology)

[the legendary ice man]

Cisco also have a 1TB device - but I couldn't find it or remember what the hell it did.

Everything is becoming much faster and it's mostly to do with fibre optic links.

[Fazares]

Well well - first of all, welcome to the forum, good first post. 8)


Hey all, this is my first post, although I've been reading these boards for several months. Basically, I have a simple, perhaps dumb question.

First, the basis of my question is this: based on M$ specs, Xbox 360 claims to have 1 teraflop (per specs on Tom's hardware). Assuming that is correct, if you have 10 million units, and you could use 1/4 of that idle processing power, you would have 250 million teraflops :shock: of processing power waiting around to be used.

MS' terraflop numbers are meaningless marketing numbers - The tri-core CPU itself will have a little over 100 GFLops of power, and as for the rest it's all coming from stuff (like the GPU) that couldn't possibly ever contribute to any sort of distributed computing tasks. To understand what framework Microsoft is working from when they give their FLop-ratings, go check out the flop ratings for the original XBox, and you'll now immediately how to decipher the new numbers.


I assume Sony will come in at at least 1 teraflop now that MS has claimed to hit the number. (I'm not sure if M$ is double counting GPUs and CPUs and such, but let's assume for a moment that the number is legit, disregarding whatever mental gymnastics M$ is using to come up with that number). If you then assume that number is inflated by a factor of four, you still have 250 gigaflops of capacity per unit. So if that is true, assuming ten million units "available" using my 25% guesstimate, idle capacity would still equate to a whopping 62.5 million teraflops. :shock: Obviously this also assumes 10 million broadband connections for each unit, and that some of this capacity could be tapped into, a big assumption.

Anyhoo, the latest list of supercomputers I saw had Blue Gene at number 1 in 2004 and it clocked in at 70.72 teraflops. Is there any way that distributed computing networks, like Climateprediction.net or SETI could tap into that idle capacity? :idea: Or is it just impossible in that there is no OS in these systems that could handle a distributed computing platform? I.e., is there no way to install software that can use the idle capacity since this thing isn't running Windows or whatever?


This is a good question, but I have to doubt that it would occur, as most users would be hesitant to farm their console out, and most companies would be scared of trying to convey the notion of even being able to to the consumer. Still, I wouldn't doubt Sony might indeed have some sort of 'grand network' scheme brewing; it's just I personally can't conceive of what it would be.


In a similar vein, of more interest to me as a gamer, my understanding is that the whole design of Cell is centered around being able to tap into the power of other cells through a network of Cells to ramp up its power using a parallel/distributed computing model. Is this possible/and or likely that each PS3 could tap into other PS3s over the internet to enhance and multiply the performance of each console? Or is that just a fantasy? :roll: Anyway, I'm no computer engineer, and I'm just doing some back of the napkin math here, so I'd appreciate it if someone could enlighten me. But strikes me that these consoles would leave a hell of a lot of cheap raw power just sitting around waiting to be tapped. No :?:



My answer above best addresses this, and specifically as it applies to gaming, I just don't see how it would be possible, as network latencies would essentially render any available Cells on the network meaningless for real-time 'need it now' power.


Off topic

M$ completely uninformative Mtv special sucked. The ourcolony commercial was at least somewhat helpful. The Tom's Hardware review is really helpful. It's posted elsewhere, I'd give credit for the link but if I hit the back button, I'll lose my post...

Resume topic.

Thanks anyone for your input.

Yeah, the MTV special was not the best.
xboxdestroya,the 1tflop figure probly comes from simple and stupid single precision ops...not very meaning...but so they did... :lol:
the massive thing coul be more feasible with ps3 and his cell processor...but not the whole ps3 cuz the gpu isnt able to connect for distributed computing networks...

[Schmeh]

xboxdestroya,the 1tflop figure probly comes from simple and stupid single precision ops...not very meaning...but so they did... :lol:

The 1TFlop refers to the system as a whole. The CPU weighs in, I believe around 120GFlops and the GPU makes up the rest.

the massive thing coul be more feasible with ps3 and his cell processor...but not the whole ps3 cuz the gpu isnt able to connect for distributed computing networks...

Although the idea of a network of PS3's working together is a great one, It is not practical. You would need to be able to reach atleast 1Gbps transfer speeds and more than likely you would need to go faster, which isn't possible since the PS3 has 1Gbps ports. Even if 1Gbps was enough, nowhere in the US do they even offer speeds around 100Mbps right now and in a couple of years when they do, I doubt enough people will have them to it practical.

[Bocuma]

"You would need to be able to reach atleast 1Gbps transfer speeds"

why would you need such a connection?
Latency would be a bigger factor

[the legendary ice man]

My school does Distributed Networking at 100mbps absolutely fine.

Even though the network is desgined to be 1Gbps, the core switch sort of...died...

anyway, What's so farfetch'd about it only existing on local networks - some people have more than one of the same console.

I have a PS2 and my bro has a PStwo.

[KKS]

Well, if folding @ home for ps3 gets developed, it will be wild! :lol:

[koldfuzion]

Hey all thanks for the interesting comments. I'm still a little confused the network is the major bottleneck, (the only thing I'm stuck on is that for something like SETI where things are sent out in units, processed and sent back, I don't see network throughput being a big deal) but I think I see the general point.

Anyhoo, thanks again.

In a similar vein, and I think on the same topic, can someone enlighten me as to what on earth exactly Ken K. is talking about in this interview quoted below? Is he just referring to in-home networking? See the comments surrounded by the :? :? faces which in particularl have me completely lost. Sony has been referring to this sort of thing for a while, but I'm still not clear on what they have in mind.

It sounds like they plan on having centralized storage and distributed processing, to improve picture quality and a whole bunch of other stuff, but it still doesn't make much sense to me in terms of what exactly he's planning to implement here. If this not a network of cells over the internet using PS3 for processing what is it? Perhaps not for distributed computing (like SETI or folding@home), but what's the general concept here? Perhaps it's limited to in-home networking but it doesn't sound that way at all to me.

It sounds to me like cell is not intended to be implemented in a vacuum inside the PS3 at a minimum, but I can't seem to tell if it's supposed to just interact with other cells in an in-home network (like the one in a new Sony 1080p HDTV--damn you Sony now I have to wait another year!), or with cells worldwide and in what way. And what is the purpose of "Cell Storage". Sounds to me like Ken has much bigger plans for Cell than just a gaming console. :shock: I'm almost waiting for him to go :evil: hoohoohahahahahahahahhhoooohaahahahah! :evil: Or maybe this is just a bunch of hype?



Honda: You used to talk about the concept of CELL or PS3 with grid computing as one of analogies. Since then it's been disclosed what kind of processor CELL actually is. How does the computing model of CELL that has networking functions by itself translate into a game console in future?

Kutaragi: In the next spring, CELL will start to enter homes as the form of PS3. Thought in it there are 7 SPEs, it may be seen only as one hardware from users. But since PS3 has Gigabit Ethernet from day one, you can connect CELLs right away at 1Gbps when you have more CELLs in your home. It can be a home server, or more PS3s, in various styles CELLs in home can connect with one another via network. With 100Mbps bandwidth, they can be connected even through internet.


Honda: Is PS3 itself rigged with such mechanism for CELL computing?


:? Kutaragi: Of course it'll be implemented in the Cell OS that's running on PS3. As you know CELL has a secure execution layer in the processor level, it has high security even in the case where they are connected through a network. The world where power of CELL connecting with one another through a network - we prepare it properly and it's implemented in PS3... :?

Honda: It seems PS3 is designed and has the spec that can naturally exist in home as an AV appliance.

Kutaragi: It's of course. Because PlayStation is not a game console. I have never remarked that PlayStation is a game console. These days a display becomes fixed-pixel so there'll be more displays for home use that can output full HD in the real resolution. Thus it's natural move that it has 2 1080p outputs.

Honda: In the aspect of AV entertainment, what function does CELL computing that centers at PS3 offer us?


Kutaragi: Imagine we can realize the world where can easily use supercomputing power in home. CELL can realize it. :? PS3 and other CELL-embedded machines have the base for CELL computing through network. :? What's possible with a supercomputer?


For example, we internally call it as "ripening", if you lay a content in a storage server on a network, CELL ripens it when it's not used and improves its quality. For example it can ripen an SD resolution movie to up-convert to an HD movie with more details. :? Those contents are stored in "CELL Storage" that is a network stroage with Gigabit Ethernet and RAID. We have many plans but will certainly offer CELL Storage. :?


(Honda's comment: the same kind of upconverting a movie was demonstrated by Intel but they explained it requires a future many-core CPU. CELL will be able to do it and other wonderful things on a home network)


Kutaragi: In future CELL home server will be in home and it'll automatically maintain contents in CELL Storage. With the security function in CELL, you can rip a copyrighted material such as DVD and put it on a storage then can ripen it to be more beautiful.

Honda: When the era of digital contents comes in the mainstream, a home network will be filled with digital data. For example you can't sort digital camera pictures by yourself because they are too many. TV programs and music are like that too. How to maintain directories for those contents and how to search them may be solved by image analysis. Moreover, the part that visualizes results for analysis and search can be navigated plainly with 3D graphics in PS3 as a frontend.

Kutaragi: Don't worry, as it's what I want most, we'll start the development soon after we release PS3. Technologies for information analysis of movie, still picture, and sound have a tremendous number of methods including research projects not well known. With supercomputer power we can apply them in home.