yoshaw
10-22-2006, 11:35 PM
PS3 EyeToy Enigma
SCE's EyeToy mastermind Richard Marks disconfirms everything known about the PS3's camera.
by Nix
October 20, 2006 - EyeToy was a significant success for Sony Computer Entertainment -- its American hype has cooled since launch, but the camera is very popular in Europe and still gets use every now again in PS2 games worldwide -- and the company is eager to explore the avenues opened by the PlayStation camera accessory. With PS3, EyeToy seemed to be poised to be a major part of the system -- at E3 2005, EyeToy figured in greatly to the amazing "Ducks, Boats & Cups" demo, and the following year, SCE demonstrated EyeToy use in Eye of Judgment and Singstar.
Despite this momentum, some answers regarding SCE's plan for EyeToy were not made as clear as we would have expected. The camera is not packed in with the system, for one, relegating the feature to the realm of accessories instead of instant mass-audience use. And SCE has made no announcements on release plans for an EyeToy outside of Japan (where Eye of Judgment is planned as a launch timeframe game.)
At SCE's Gamers Day, we were able to corner EyeToy designer and spearhead promoter Richard Marks about the EyeToy. (It wasn't easy to keep him in that corner, by the by -- Richard was here strictly as a PlayStation 3 feature guru, but because the press recognized him instantly for his EyeToy credentials, Richard was stuck trying to come up with answers and non-answers for things he wasn't allowed to talk about or there to present ... later in the evening, we think we caught him wandering the show with somebody else's nametag on to help avoid questions he couldn't talk about.) What's up with EyeToy on PS3, we asked? His answers to our questions led us to the understanding that maybe there is no EyeToy for PS3...
That statement requires clarification (especially for all the EyeToy fans freaking out right now that there will never be a sequel to EyeToy.) First off, the PS3 camera that will launch with Eye of Judgment is not necessarily EyeToy -- it's a camera to be used for that game, as well as perhaps other features. That EyeToy HD IP camera shown at E3? That's not necessarily the EyeToy either -- it's an EyeToy, but that model was really just a demonstration version designed to show off high-def video recognition and introduce the idea of an IP-based PS3 camera. OK, so the real PS3 EyeToy camera will be the amazing infra-red camera (http://www.gamespy.com/articles/584/584744p1.html) displayed at DICE 2005? Again, no -- that concept is of course something SCE is toying with, but actually delivering it is something that ... well, it brought a waned but sly smile to Richard's face when we asked.
At this monment then, there is no specific PS3 EyeToy on the release schedule or to demonstrate. And yet, PS3 will have camera integration built right in at launch and also Video Chat as one of its major features. What's up with that? Well, the PS3 is a USB-friendly console, and that includes the classic EyeToy as well as probably other PC cameras. The system's ready, and in fact SCE has a variety of cameras ready ... it just doesn't have an exact product called "EyeToy" right now.
EyeToy itself, then, is still something that's coming to PS3. In various ways, it's already here, in games like Eye of Judgment and Singstar. EyeToy is really just a concept rather than a product line -- it is gaming software that captures the world and brings it into your games. Previously, it took a specific piece of hardware for "EyeToy", and there should be a specific PS3 EyeToy device when the company is ready to make that push, but the PS3 is capable of using cameras without them being labeled EyeToy.
Once SCE begins to make moves on the camera and EyeToy end, look for greatly-expanded abilities over that already seen on PS2. Even though there was no "official EyeToy" to speak of, Richard did explain that because the PS3 could accept a data flow from a camera completely uncompressed in realtime, it's technically free on the processor to bring that data into the system and perhaps use in a game. (Actual recognition procession of course takes the machine to do, but PS3's EyeToy applications could be much more robust than ever on PS2, and EyeToy can be used in games no longer have to be just camera-based minigames.) Richard also answered that it would not be unreasonable to expect the PS3's camera abilities to allow users to record videos to the HDD in given modes.
We weren't able to pull much more explanation out of Mr. Marks (and we're not sure he wanted us to go into as much detail as we did about things he could hardly talk about on the record), but he did indicate that we will be hearing more about EyeToy not too long after launch -- once the focus is off the hardware and the launch games, more should be open for discussion on the rest of the PS3's camera abilities. We've been eager for lots more info about PS3 plans for EyeToy for some time now, and hope to be able to pull up that info shortly after the release of the system.
http://ps3.ign.com/articles/740/740684p1.html
Now that you've read the non-proprietary nature of PS3's camera functionality and usage. Here's a mind crunching thing I found that I had no idea existed until today. Do read it in full and if you're like me learning about it for the first time, You'll start patting yourself on the back for being entrusting your gaming needs into a company that is as much devoted to changing the face of gaming as it did with PS1's 2D to 3D revolution.
Read on fellas :cheers: and praise be on Dr.Richard Marks!
DR. RICHARD MARKS INFRA-RED EYETOY PRESENTATION AT DICE 2005
Mind-Numbing New Interface Technologies
Dr. Richard Marks, one of the creators of the Eye-Toy, talks about new directions in interface design.
By Dave Kosak | Feb. 1, 2005
At first it didn't look like much: Dr. Richard Marks, who helped create Sony's awesome Eye-Toy peripheral, walked us through some PowerPoint slides talking about different game interfaces. Then things got crazy.
Game interfaces often get the shaft. Developers invest tons in graphics or sound or physics, but nobody ever invests tons of time into the control scheme. Often people don't even really think about it. Game controllers are ubiquitous. You push a button, and stuff happens. But the interface is literally a player's connection to the game: it's where the action happens.
And after a couple decades of stagnation (game pads are basically the same as they were for the NES, save for more buttons) we're finally on the verge of some crazy new technologies for how people interact with games. The Eye-Toy is leading the way, proving that there's an opportunity for new concepts to hit mainstream.
http://xs208.xs.to/xs208/06420/dice_2005_interface1_1107325674.jpg
Dr. Marks shows off a 'Minority-Report' style interactive interface
This was all just lip-service until Dr. Marks started firing up demos and showing what new technology was on the horizon. Here's what's coming up, including some cool stuff that I never imagined was so close to reality:
Head Tracking: Head-tracking works with the current Eye-Toy, and it's been successfully applied to Eye-Toy: AntiGrav, where you'd move your head and arms to steer a snowboarder. But there are other applications, as well. One demo was of a 3D instant replay: Dr. Marks brought up a freeze-frame of a soccer match. Then, as he moved his head back and forth, the scene rotated accordingly, showing the action from multiple angles like in The Matrix and swinging the camera around with his head in real-time.
This technology could be very cool when applied to first-person shooters. Imagine that you creep up to the corner of an alley. Then, when you want to peek around the corner, instead of strafing your character over you could just move your head and literally 'peek.' Dr. Marks showed a demo of a street scene where he moved his head to look down a street and then moved it back to duck around cover as the bullets flew.
Unfortunately, head-tracking technology uses up about 20% of the CPU with the current generation of game platforms. No first-person game developer would want to do that. But in he next generation of hardware, the processors are many times more powerful and the drain on system resources is much more trivial. Expect to see some cool things!
The 'Minority Report' Demo: Okay, most of us saw the movie. Tom Cruise put on special gloves and then played with a computer interface by literally dragging and dropping folders and manipulating data with his hands. Guess what? The technology is just about there, and even though it's still rooted to a TV screen, at least you don't need the goofy gloves.
The secret is in "retro-reflective" material. This material takes low light (say, the dim LED on the front of the Eye-Toy camera) and reflects 100% of it back at the lens. The upshot is that the camera can track specific objects with nearly pixel-perfect accuracy. Dr. Marks picked up a small rubber doohickey that he held in his hand coated with a retro-reflective surface. Then, he opened up an on-screen photo gallery. Using the device, he could reach around the screen to select individual photos. He could "grab" them by squeezing the device (the demo made a satisfying 'click!') and drag them around the screen. He could even grab them and then rotate them. The rotation was a little dodgy (sometimes it took two tries), but it worked, and it worked using technology that's pretty much available to consumers today. I knew it: we live in the future.
http://xs208.xs.to/xs208/06420/dice_2005_interface2_1107325899.jpg
Next-gen cameras can easily and accurately map your movements onto a digital avatar in 3D space.
Future Cameras and "Per-Pixel Distance Measurement." I didn't really 'get' this technology until Dr. Marks showed the demo. Here's the deal: current camera interfaces (like the Eye-Toy) can track movement. You wave your arms in front of the camera, and it can track the waving motion. But if someone stands behind you and jumps around, it'll track that motion, as well. And if you stand in all black against a black background, forget it: no matter how much you move, the camera can't see you.
But the next generation of camera interfaces can measure the actual distance to objects using infra-red pulses. And they're extremely precise. They're able to trace the exact contour of any shape, and they can track it as it moves toward or away from the camera. This changes everything!
So, for example, if the camera is set up to only track objects within a certain distance to the camera, you can reach forward to manipulate game objects, then pull your hand back to stop interacting. More importantly, some guy can stand behind you dancing around and the camera will just ignore it: he's too far away.
Sure, that's cool. But once the camera knows how close or far away you are, it opens up other possibilities, specifically in the way that digital objects interact with the real world depicted on the screen. In another demo, Dr. Marks held up a wand that attracted butterflies that swirled around it on the screen. As he moved the wand, the butterflies flew to chase it. But then, when he passed the wand behind his back, the butterflies on the screen flew behind him. It really looked like they were flying around him. The illusion was much more complete than you can get with today's technology.
Hold up, he hasn't even started. Cameras with this kind of resolution can do real-time motion capture. So, you can dance in front of the camera, and all of your movements can be tracked and then applied to a digital model rendered on the screen. In his next demo, Dr. Marks moved around and on the screen a skeletal version of himself moved to match. He'd wave his arms and the skeleton would do the same. Physics was built into the simulation, so when he punched his arms forward, the skeleton punched, and it could hit objects around the virtual room. Because the camera was tracking distances, it could actually track where he was in a 3D space -- standing in certain spots triggered certain actions, for instance. The Eye-Toy's motion tracking looks pretty primitive in comparison. Imagine the gaming possibilities of this kind of interface! You'd literally be, full body, involved in the on-screen action, stepping into another character.
New Interfaces for Handhelds. Heldheld gaming is in the midst of a revolution. Kirby's Tilt'n'Tumble could be interacted with by tilting the game console. The Nintendo DS has a touch-screen, which is super-intuitive. People are really starting to experiment!
Adding a camera to a handheld can open up tons of possibilities. Dr. Marks, who used the PSP in all his examples, is particularly excited about the idea of using a handheld as a "lens" to view the world. In other words, stick a camera on top of the PSP facing out, then look "through" it (at your screen) at the real world but with digital stuff overlaid on top of it.
http://xs208.xs.to/xs208/06420/dice_2005_interface3_1107326030.jpg
Dr. Marks walked the audience through several demos
The Freaky Part
His example completely blew me away. Imagine being able to set a retro-reflective marker onto your coffee table, something that the game can track. Now, you point the camera at it and look through your game console screen and 3D characters could be imaged right on top of the real-time picture of your coffee table. Characters in a fighting game, for example, could be squaring off on your furniture. You could stand up, move around, and look at the battle from different angles: the characters would all be rendered appropriately. In a multiplayer game, you could have several players standing around the coffee table all looking at different views of the same 3D combat.
It sounds crazy.
But the crazy part is, this isn't some sort of weird hypothetical future scenario. The technology is pretty much here. This is being worked on right now. By the end of the next console generation, per-pixel distance-measurement cameras will be available at a consumer price point.
Now, I don't know how long it'll be before you look through your PSP and see the Soul Calibur characters bouncing up and down on your bedsheets, but we're hella closer than I thought.
http://www.gamespy.com/articles/584/584744p1.html
:shocked: To imagine such an innovative technology under development almost 2+ years before PS3 ships anywhere in the world. I don't know what you guys think about Dr. Marks, but I believe he's going to create technology for PS3 that's gonna make Minority Report possible in our lifetime, lol.
And hey, it's not like the thing is out of the question. As you can witness via the gamespy Dice2005 article above. It was "happening" onstage in Feb 2005 and the author witnessed it for real. It's coming and I'd like to wager Sony was waiting for PS3 and its processing power to help this goal become reality 0_0
Pat yourselves on the back gentlemen. For you've made a logical and excellent choice in supporting a company taking strides in uncharted directions.
Respect! :worthy:
+rep Dr. Richard Marks!
SCE's EyeToy mastermind Richard Marks disconfirms everything known about the PS3's camera.
by Nix
October 20, 2006 - EyeToy was a significant success for Sony Computer Entertainment -- its American hype has cooled since launch, but the camera is very popular in Europe and still gets use every now again in PS2 games worldwide -- and the company is eager to explore the avenues opened by the PlayStation camera accessory. With PS3, EyeToy seemed to be poised to be a major part of the system -- at E3 2005, EyeToy figured in greatly to the amazing "Ducks, Boats & Cups" demo, and the following year, SCE demonstrated EyeToy use in Eye of Judgment and Singstar.
Despite this momentum, some answers regarding SCE's plan for EyeToy were not made as clear as we would have expected. The camera is not packed in with the system, for one, relegating the feature to the realm of accessories instead of instant mass-audience use. And SCE has made no announcements on release plans for an EyeToy outside of Japan (where Eye of Judgment is planned as a launch timeframe game.)
At SCE's Gamers Day, we were able to corner EyeToy designer and spearhead promoter Richard Marks about the EyeToy. (It wasn't easy to keep him in that corner, by the by -- Richard was here strictly as a PlayStation 3 feature guru, but because the press recognized him instantly for his EyeToy credentials, Richard was stuck trying to come up with answers and non-answers for things he wasn't allowed to talk about or there to present ... later in the evening, we think we caught him wandering the show with somebody else's nametag on to help avoid questions he couldn't talk about.) What's up with EyeToy on PS3, we asked? His answers to our questions led us to the understanding that maybe there is no EyeToy for PS3...
That statement requires clarification (especially for all the EyeToy fans freaking out right now that there will never be a sequel to EyeToy.) First off, the PS3 camera that will launch with Eye of Judgment is not necessarily EyeToy -- it's a camera to be used for that game, as well as perhaps other features. That EyeToy HD IP camera shown at E3? That's not necessarily the EyeToy either -- it's an EyeToy, but that model was really just a demonstration version designed to show off high-def video recognition and introduce the idea of an IP-based PS3 camera. OK, so the real PS3 EyeToy camera will be the amazing infra-red camera (http://www.gamespy.com/articles/584/584744p1.html) displayed at DICE 2005? Again, no -- that concept is of course something SCE is toying with, but actually delivering it is something that ... well, it brought a waned but sly smile to Richard's face when we asked.
At this monment then, there is no specific PS3 EyeToy on the release schedule or to demonstrate. And yet, PS3 will have camera integration built right in at launch and also Video Chat as one of its major features. What's up with that? Well, the PS3 is a USB-friendly console, and that includes the classic EyeToy as well as probably other PC cameras. The system's ready, and in fact SCE has a variety of cameras ready ... it just doesn't have an exact product called "EyeToy" right now.
EyeToy itself, then, is still something that's coming to PS3. In various ways, it's already here, in games like Eye of Judgment and Singstar. EyeToy is really just a concept rather than a product line -- it is gaming software that captures the world and brings it into your games. Previously, it took a specific piece of hardware for "EyeToy", and there should be a specific PS3 EyeToy device when the company is ready to make that push, but the PS3 is capable of using cameras without them being labeled EyeToy.
Once SCE begins to make moves on the camera and EyeToy end, look for greatly-expanded abilities over that already seen on PS2. Even though there was no "official EyeToy" to speak of, Richard did explain that because the PS3 could accept a data flow from a camera completely uncompressed in realtime, it's technically free on the processor to bring that data into the system and perhaps use in a game. (Actual recognition procession of course takes the machine to do, but PS3's EyeToy applications could be much more robust than ever on PS2, and EyeToy can be used in games no longer have to be just camera-based minigames.) Richard also answered that it would not be unreasonable to expect the PS3's camera abilities to allow users to record videos to the HDD in given modes.
We weren't able to pull much more explanation out of Mr. Marks (and we're not sure he wanted us to go into as much detail as we did about things he could hardly talk about on the record), but he did indicate that we will be hearing more about EyeToy not too long after launch -- once the focus is off the hardware and the launch games, more should be open for discussion on the rest of the PS3's camera abilities. We've been eager for lots more info about PS3 plans for EyeToy for some time now, and hope to be able to pull up that info shortly after the release of the system.
http://ps3.ign.com/articles/740/740684p1.html
Now that you've read the non-proprietary nature of PS3's camera functionality and usage. Here's a mind crunching thing I found that I had no idea existed until today. Do read it in full and if you're like me learning about it for the first time, You'll start patting yourself on the back for being entrusting your gaming needs into a company that is as much devoted to changing the face of gaming as it did with PS1's 2D to 3D revolution.
Read on fellas :cheers: and praise be on Dr.Richard Marks!
DR. RICHARD MARKS INFRA-RED EYETOY PRESENTATION AT DICE 2005
Mind-Numbing New Interface Technologies
Dr. Richard Marks, one of the creators of the Eye-Toy, talks about new directions in interface design.
By Dave Kosak | Feb. 1, 2005
At first it didn't look like much: Dr. Richard Marks, who helped create Sony's awesome Eye-Toy peripheral, walked us through some PowerPoint slides talking about different game interfaces. Then things got crazy.
Game interfaces often get the shaft. Developers invest tons in graphics or sound or physics, but nobody ever invests tons of time into the control scheme. Often people don't even really think about it. Game controllers are ubiquitous. You push a button, and stuff happens. But the interface is literally a player's connection to the game: it's where the action happens.
And after a couple decades of stagnation (game pads are basically the same as they were for the NES, save for more buttons) we're finally on the verge of some crazy new technologies for how people interact with games. The Eye-Toy is leading the way, proving that there's an opportunity for new concepts to hit mainstream.
http://xs208.xs.to/xs208/06420/dice_2005_interface1_1107325674.jpg
Dr. Marks shows off a 'Minority-Report' style interactive interface
This was all just lip-service until Dr. Marks started firing up demos and showing what new technology was on the horizon. Here's what's coming up, including some cool stuff that I never imagined was so close to reality:
Head Tracking: Head-tracking works with the current Eye-Toy, and it's been successfully applied to Eye-Toy: AntiGrav, where you'd move your head and arms to steer a snowboarder. But there are other applications, as well. One demo was of a 3D instant replay: Dr. Marks brought up a freeze-frame of a soccer match. Then, as he moved his head back and forth, the scene rotated accordingly, showing the action from multiple angles like in The Matrix and swinging the camera around with his head in real-time.
This technology could be very cool when applied to first-person shooters. Imagine that you creep up to the corner of an alley. Then, when you want to peek around the corner, instead of strafing your character over you could just move your head and literally 'peek.' Dr. Marks showed a demo of a street scene where he moved his head to look down a street and then moved it back to duck around cover as the bullets flew.
Unfortunately, head-tracking technology uses up about 20% of the CPU with the current generation of game platforms. No first-person game developer would want to do that. But in he next generation of hardware, the processors are many times more powerful and the drain on system resources is much more trivial. Expect to see some cool things!
The 'Minority Report' Demo: Okay, most of us saw the movie. Tom Cruise put on special gloves and then played with a computer interface by literally dragging and dropping folders and manipulating data with his hands. Guess what? The technology is just about there, and even though it's still rooted to a TV screen, at least you don't need the goofy gloves.
The secret is in "retro-reflective" material. This material takes low light (say, the dim LED on the front of the Eye-Toy camera) and reflects 100% of it back at the lens. The upshot is that the camera can track specific objects with nearly pixel-perfect accuracy. Dr. Marks picked up a small rubber doohickey that he held in his hand coated with a retro-reflective surface. Then, he opened up an on-screen photo gallery. Using the device, he could reach around the screen to select individual photos. He could "grab" them by squeezing the device (the demo made a satisfying 'click!') and drag them around the screen. He could even grab them and then rotate them. The rotation was a little dodgy (sometimes it took two tries), but it worked, and it worked using technology that's pretty much available to consumers today. I knew it: we live in the future.
http://xs208.xs.to/xs208/06420/dice_2005_interface2_1107325899.jpg
Next-gen cameras can easily and accurately map your movements onto a digital avatar in 3D space.
Future Cameras and "Per-Pixel Distance Measurement." I didn't really 'get' this technology until Dr. Marks showed the demo. Here's the deal: current camera interfaces (like the Eye-Toy) can track movement. You wave your arms in front of the camera, and it can track the waving motion. But if someone stands behind you and jumps around, it'll track that motion, as well. And if you stand in all black against a black background, forget it: no matter how much you move, the camera can't see you.
But the next generation of camera interfaces can measure the actual distance to objects using infra-red pulses. And they're extremely precise. They're able to trace the exact contour of any shape, and they can track it as it moves toward or away from the camera. This changes everything!
So, for example, if the camera is set up to only track objects within a certain distance to the camera, you can reach forward to manipulate game objects, then pull your hand back to stop interacting. More importantly, some guy can stand behind you dancing around and the camera will just ignore it: he's too far away.
Sure, that's cool. But once the camera knows how close or far away you are, it opens up other possibilities, specifically in the way that digital objects interact with the real world depicted on the screen. In another demo, Dr. Marks held up a wand that attracted butterflies that swirled around it on the screen. As he moved the wand, the butterflies flew to chase it. But then, when he passed the wand behind his back, the butterflies on the screen flew behind him. It really looked like they were flying around him. The illusion was much more complete than you can get with today's technology.
Hold up, he hasn't even started. Cameras with this kind of resolution can do real-time motion capture. So, you can dance in front of the camera, and all of your movements can be tracked and then applied to a digital model rendered on the screen. In his next demo, Dr. Marks moved around and on the screen a skeletal version of himself moved to match. He'd wave his arms and the skeleton would do the same. Physics was built into the simulation, so when he punched his arms forward, the skeleton punched, and it could hit objects around the virtual room. Because the camera was tracking distances, it could actually track where he was in a 3D space -- standing in certain spots triggered certain actions, for instance. The Eye-Toy's motion tracking looks pretty primitive in comparison. Imagine the gaming possibilities of this kind of interface! You'd literally be, full body, involved in the on-screen action, stepping into another character.
New Interfaces for Handhelds. Heldheld gaming is in the midst of a revolution. Kirby's Tilt'n'Tumble could be interacted with by tilting the game console. The Nintendo DS has a touch-screen, which is super-intuitive. People are really starting to experiment!
Adding a camera to a handheld can open up tons of possibilities. Dr. Marks, who used the PSP in all his examples, is particularly excited about the idea of using a handheld as a "lens" to view the world. In other words, stick a camera on top of the PSP facing out, then look "through" it (at your screen) at the real world but with digital stuff overlaid on top of it.
http://xs208.xs.to/xs208/06420/dice_2005_interface3_1107326030.jpg
Dr. Marks walked the audience through several demos
The Freaky Part
His example completely blew me away. Imagine being able to set a retro-reflective marker onto your coffee table, something that the game can track. Now, you point the camera at it and look through your game console screen and 3D characters could be imaged right on top of the real-time picture of your coffee table. Characters in a fighting game, for example, could be squaring off on your furniture. You could stand up, move around, and look at the battle from different angles: the characters would all be rendered appropriately. In a multiplayer game, you could have several players standing around the coffee table all looking at different views of the same 3D combat.
It sounds crazy.
But the crazy part is, this isn't some sort of weird hypothetical future scenario. The technology is pretty much here. This is being worked on right now. By the end of the next console generation, per-pixel distance-measurement cameras will be available at a consumer price point.
Now, I don't know how long it'll be before you look through your PSP and see the Soul Calibur characters bouncing up and down on your bedsheets, but we're hella closer than I thought.
http://www.gamespy.com/articles/584/584744p1.html
:shocked: To imagine such an innovative technology under development almost 2+ years before PS3 ships anywhere in the world. I don't know what you guys think about Dr. Marks, but I believe he's going to create technology for PS3 that's gonna make Minority Report possible in our lifetime, lol.
And hey, it's not like the thing is out of the question. As you can witness via the gamespy Dice2005 article above. It was "happening" onstage in Feb 2005 and the author witnessed it for real. It's coming and I'd like to wager Sony was waiting for PS3 and its processing power to help this goal become reality 0_0
Pat yourselves on the back gentlemen. For you've made a logical and excellent choice in supporting a company taking strides in uncharted directions.
Respect! :worthy:
+rep Dr. Richard Marks!