PING Platform

Proof of Concept 2: Exploring Multitouch

admin — Tue, 18 Nov 2008 04:32:58 +0000

So, as mentioned in our previous post, we’re taking a step in a slightly new direction, and exploring multitouch interaction. This has obviously been an area of extreme growth in the last few years, with products from Apple and Microsoft making use of multitouch interfaces, just to name a very few. We’re most definitely standing on the shoulders of giants in this arena, particularly in gestural libraries. What this means is that we’re able to explore what this can mean in the context of casual public gaming. We’ve also realized that a much more interesting investigation involves the mixture of multi-touch gestures with motion in real 3d space. We hope that by marrying our two prototypes, we can provide a much more interesting platform upon which to develop.

A quick introduction for now, with more to come very soon: For a second proof, we’re building a general purpose multi-touch screen for the Pervasive Technology Labs. It’s a proof to show ourselves that we can do it, plus an opportunity to test some interaction design theories. We’re following someone else’s instructions, which are based on the proven methodology of FTIR. It’s 3×4 aspect ration (18″x24″), and will initially be tested using the fantastic tbeta software. Pictures, findings and construction details to follow soon.

Evaluation and Shifting Focus

tonydewan — Thu, 30 Oct 2008 04:59:21 +0000

In the past several weeks, alot has happened. First and foremost we’ve updated and improved our proof of concept. We have also had the opportunity to show it to many different people, not the least of which was Camille Utterback, who was in town for lecture. We got some great feedback all around. Here is what we learned:

Controls aren’t accurate enough – Despite the fantastic smoothing algorithm that Chris built, the controls just weren’t accurate enough to provide solid game play. Sure, you can easily play a game of low pressure pong, but any long term game play makes the limits of the interface apparent.
Interface paradigm isn’t obvious – In observing many different users attempt to interact with the system, it seems to be too difficult to master without instruction. This adds unnecessary complexity, particularly as we view our work in the context of casual public gameplay.
We were too focused on the deliverable - We’ve had very lofty goals from the beginning for this project, particularly when we’re talking about our deliverable. We realized that maybe we should use the time and resources we have to focus more on exploration and have confidence that we’ll craft a strong deliverable eventually.

So, we’ve decided that infrared-enabled edges add unneeded complexity and are innacurate. Well, being that we had planned for a multi-touch environment in the future, and given that that particular area of HCI and technology has exploded in the past few years, we decided to start exploring that alternative. But, as you’ll see in a future post, that whole multi-touch project thing is already pretty saturated. What will make us different? Well, two things:

Game platform – everything we have yet to find isn’t really focused on the gaming niche. There are quite a few platforms or libraries to build a multi-touch table. But none of them, that we’ve seen, really makes it easy to make quick social games. Which brings us to;
Built in ActionScript – with the advent of AS3, the Flash/Flex/AIR environment became a strong framework for building intensive applications. And, given that much of what is necessary to bring multi-touch to Flash was possible in AS2, the speed and stability improvements that come with AS3 make this a strong alternative. Most importantly, games can much more easily be developed than in a lower level environment.

Best of all, Chris has been developing some strong video processing algorithms in Flex/ActionScript 3 in his work with the PTL, so we’re a few steps ahead. We’ve also got quite a bit of robust research and real world examples to guide our work.

So, to wrap up: we’re shifiting focus to be more explorative, particularly in the area of multi-touch interactions. Also, in the interest of academic honesty and intellectual humility, we’ll shortly be writing a post that rounds up all of the multi-touch/table installations, frameworks, projects and ideas we’ve found, plus analyzing and evaluating them.

Proof of Concept: Wrap Up and Next Steps

tonydewan — Thu, 09 Oct 2008 14:39:36 +0000

Wrap Up

Our proof of concept is complete! Well, sort of. Here is what we originally set out to do with the month of October:

According to the schedule we have tentatively set, our plan is to create a small scale proof of concept by October 1st, 2008

Well, we actually created a full-scale proof of concept by October 8, 2008. It meets and exceeds all of the requirements we made in the beginning. However, our next tentative goal was to build a full scale proof by November 1. What we have currently achieved doesn’t meet our goals for that deliverable in a few ways:

Only 2 Player
We currently have built and developed just 2 sides of the PING Platform table. We want all 4 in a full scale proof.
Only Uses Long Range Sensors
During the very beginning of the proof phase, it became quickly apparent that we would need to come up with some way to adjust for the 7-8″ dead zone provided by each sensor. We originally thought we could compensate for this with the opposing sensor. However, that is less than idea. So, we realized we would need a much shorter range sensor at each corner, in addition to the aforementioned long range sensor.

So, this leads us to…

Next Steps

What are we going to do next? Well, there are quite a few things we can work on that are independent of our quest for funding. They are:

Complete (4-sided) Full Scale Proof

Allow all four sides of the table to be interactive.
Add Short Range Sensors

We have one short range sensor to test with. We’ll need to adapt the software to handle multiple sensors at each corner.
Software Platform Expansion

There are several important features of the software platform API that need to be completed before we can release it and start developing the OS and games.
PingOS design/testing/development

We’ll need to flesh out several of our ideas about interface/interaction design and test them. We can obviously then develop what we build into the platform.
Game Development

After all of the above are done, we start to make and adapt games to the platform.

More info on all of these threads to come soon.

Proof of Concept: Week 5

tonydewan — Fri, 03 Oct 2008 16:25:18 +0000

Goals for This Week:

POC Wrap Up
Screen Material Research

POC Wrap Up

Week 5 was spent wrapping up the loose ends for our full scale proof, as well as prepare for the Tuesday adviser meeting. We made a lot of relatively small steps that led to big results. We spent some time more fully integrating the Pong game visually. This makes the proof feel much more complete and holistic. We also reworked some bits of the software platform to make the data more accurate. And finally, Chris built a fantastic smoothing algorithm that all but removes the jitter and small inaccuracies we were seeing in Pong. The PING Platform now feels much more like a solid, complete, playable game.

Screen Material Research

Just a quick note: we’re researching what materials might make the most sense for a rear projection screen, particularly in the context of the cramped table space.

Next Week: Proof of Concept Wrap-Up and Next Steps

Proof of Concept: Week 4

tonydewan — Mon, 22 Sep 2008 16:12:14 +0000

Goals for Week 4:

Continued data abstraction
Pong game integration
2nd player sensor mounting
Full scale proof part 1

Continued data abstraction

Up to this point, the data has been abstracted to each corner. We have now added another level of access: the side. This is ultimately what will be used most by developers, we think. The side data takes into account both corners that make up the side. This is where several of the unique aspects of our interface come into existence.

Pong Game Integration

This week, we’ll be integrating the Pong code we have with the platform. This seems like it’ll be pretty quick work. However, it’ll really bring everything together.

2nd Player Sensor Mounting

At this point, we’ve been working with one side. That is, two sensors working together to make the interface for a single user. In order to meet our requirements for a full scale prototype, we’ll be adding another side. This will allow us to actually play Pong against each other instead of against ourselves (or an AI.)

Full scale proof part 1

We’ll hopefully begin setup/construction of our full scale proof of concept this week. We’ve got the space in the Pervasive Lab, we’ve got the sensors and mounting blocks, and we we’ve got the software in a pretty good place too. Our goal for part 1 is to get the space set up and organized, so it can be ready for a software test in part 2. We’ll have pictures once it’s ready.

Goals for Week 5:

POC Wrap Up
Screen Material Research

Proof of Concept: Week 3

tonydewan — Mon, 15 Sep 2008 23:55:23 +0000

Goals for Week 3:

Continued Platform Development
Continued PingOS Design
Begin installation material research

Continued Platform Development

Chris has done some exceptional work on architecting and abstracting, and created an app that presents realtime data from the sensors in an ordered way. We’re at the point now that we should revisit what the best way to present data to a developer might be. By side? By corner? In pixels? Percentage? Physical measurement? We think we’ve essentially ironed out the best case for this, but we’ll know for sure how affective our choices are when we start developing games.

Continued PingOS Design

I’ve sketched out a few basics concepts for visualizing a grid based game selection interface. My plan is to flesh out these ideas in Photoshop, and then move forward with a few to a Flash based prototype. We can then hopefully do some usability testing using our full scale prototype. How exciting!

Begin installation material research

The biggest concern that we have in this whole operation is the actual physical installation. We plan on letting the process of finding funding help dictate some of those choices. However, there is still plenty for us to know and learn. We’ve begun the process of investigating materials and doing preliminary physical structure sketches. We’re also reexamining how we might be adjusting the affordances of our physical structure. More on this later…

Goals for Next Week:

Continued data abstraction
Pong game integration
2nd player sensor mounting

Proof of Concept: Week 2

tonydewan — Wed, 10 Sep 2008 05:06:34 +0000

Here are the tentative goals for the week. When these change, or we do more, this post will be updated.

Goals for Week 2:

POC Hardware Rig v. 1.1: Vertical Sensor Adjustment
More Software Platform Dev
PingOS Interface Research

POC Hardware Rig v. 1.1: Vertical Sensor Adjustment

Because of the problems with interference, we realized we needed to be able to adjust the sensor mounting vertically, in addition to horizontally. So, we added more vertical mounting points for the sensors (see images below) Now, we can easily adjust the height of each sensor as we’re testing. Nice! We also discussed some more in depth testing we’ll need to do once we’re out of the Proof phase. This rig will be crucial for that.

fig. 2: Chris Adjusting the Sensor Location

Fig. 1: The vertical lines of holes in the block are new mounting points.

More Platform Development

If we haven’t made this clear, we are building as much of the software platform as possible during our Proof of Concept phase. So rather than hacking out a quick solution during the Proof and then rebuilding from scratch later, we are spending the time to effectively architect the platform now. This makes our Proof phase longer, but saves us a bunch of time and headaches in the future. Even better, it also puts us in the shoes of the game developer from day one: we’ll be using the platform to build our first game. We then automatically get to test our architecture in a real world environment, and see how preforms.

So, at this point in the Proof phase, we’re working towards abstracting the structure of the software Platform. We’ll get more into this next week.

PingOS Interface Research

In an attempt to make the best use of our limited time resources, the two of us are trying to work in parallel as much as possible. I think they call that the phased project management model. Or is it spiral model? Anyway, since Chris is much more qualified and able to architect a software platform, he’s taking the lead on that thread. While that important work is happening, I’m hoping to knock out some of the preliminary work for other phases. One of those is the “OS” interface design.

PingOS? What?Alright, so this is a little awkward. PingOS is a term we’re using to describe the interface through which games will be accessed. It’s like the Dashboard on the XBox or the System Software on the PS3. As Chris pointed out, Wikipedia says that an Operating System does this:

Management and coordination of activities
Acts as a host for applications
Handles details of operating with hardware

That fits, right? Depending upon how literally you take some of those statements, that’s exactly what we’re building. Disagree? Sorry. It’s a development term, so it just might change.

This is an important part of what we’re doing, because it will provide an interaction framework for games to work from. Essentially, it involves defining the visual paradigms of this unique input concept. There will be more on this later.

Goals for Week 3:

Continued Platform Development
Continued PingOS Design
Begin installation material research

Proof of Concept: Week 1

tonydewan — Sun, 07 Sep 2008 00:04:41 +0000

This week was first and foremost about getting acclimated. We hadn’t ever hooked into the Phidgets Flex Platform. So, we did that. We also hadn’t had more than a single IR sensor to work with until this week. A new Phidgets interface kit and new sensors were graciously provided by Dr. Baker, so we were able to plug in two sensors at the same time.

We then began planning.

We planned some of the specifics of the software platform. I’m not going to get too much into detail there just yet. Suffice it to say, our goal is to make game development as easy as possible.

We also planned our testing setup for the hardware. Essentially, we needed a way to incrementally adjust sensor placement along a single axis, while still mounting both the sensor and the distance adapter. We came up with 4ft x 4in strips of peg board, blocks of wood with dowels projecting from their bottom and sensors mounted on their side. Confused? There will be pictures next week.

We put it all together, and…disaster!

We ran into our first major snag when we mounted the sensors, hooked everything together, and saw that the the sensor data was noticeably erratic. Our high precision IR sensors were all of a sudden not so high precision.

We realized that the sensors were interfering with each other.

I’m going to try to explain why. Now, this is based on some factual knowledge, some conjecture, and some observation. (If it turns out I’m a moron and way off base, and you’re an engineer for Sharp or something, please tell me. I hate living in ignorance.) So, the sensors have two main parts: an IR emitter and an IR receiver. The sensor is able to measure data by measuring the time it takes for an IR pulse from the emitter to bounce back to the reader. Our data was jittery because the emitted IR pulse from each sensor was reaching the reader of the opposite sensor.

The good news is that this was only happening when there was nothing in between the sensors. Like a hand. Well, we want hands there. In fact, we plan hands to be there a lot. That’s kind of the whole point.

So, how did we fix it?

Well, we haven’t completely fixed it yet. We have some ideas, though. First, by adjusting the sensors so that they aren’t at the exact same height, and by making sure the emitter of each sensor isn’t aimed directly at the receiver of the other, we can reduce the jitter somewhat. Also, we think that by limiting the vertical scope of the sensors with some kind of physical obstruction, we might be able to reduce it further. We are also investigating ways we might be able to filter the IR signals so that they each exist in slightly different parts of the IR spectrum. Finally, we realized that the jitter is a recognizable pattern, and we are confident that it can be filtered out in software. Hopefully, some combination of these solutions will result in an error free data source.

Goals for Week 2:

POC Hardware Rig v. 1.1: Vertical Sensor Adjustment
More Software Platform Dev
PINGOS interface research, mockups

Proof of Concept: Overview

tonydewan — Wed, 03 Sep 2008 23:58:19 +0000

According to the schedule we have tentatively set, our plan is to create a small scale proof of concept by October 1st, 2008. This will be both a hardware and software proof. We’ll be making sure we aren’t crazy, and all of the grandiose plans we’ve made are actually feasible. That means making sure the sensors work as planned; it means testing the sensor placement; it means implementing a basic version of our first game to test interface and interactions; and it means making the first steps in developing the software platform. In the coming weeks, we’ll elaborate on the each step in the process with a new blog post. I’ll provide a nice list of links below so you can follow along easily.

Platform Overview

tonydewan — Wed, 03 Sep 2008 03:23:59 +0000

The PINGplatform is a gaming platform, consisting of both hardware and software tools. Here are the basics.

Hardware

The PINGplatform makes use of the Phidgets interface board. Basically, Phidgets are “…a set of ‘plug and play’ building blocks for low cost USB sensing and control from your PC.” The Phidgets interface kit makes it really quick and easy to interface with many kinds of sensors, provided they are designed to interface with it. They have quite a few sensors to choose from.

For the first version of the PINGplatform, our user interface is designed around 8 Sharp GP2Y0A02YK infrared distance sensors. These sensors are fantastically accurate, particularly within the limited range we’re interested in. So, why 8 sensors? Well, with 2 sensors at each corner, we can measure the distance from each corner along all 4 sides of the table. This allows us to track 2 points independently along each side of the table. That means we can allow for some relatively complex interactions.

There are two sensors at each corner, 1 sensing across each table edge.

The Phidgets kit abstracts all of the hard bits, providing a robust API for working with sensor data in just about any language you might want. Which leads us to the software…

Software

Our software platform will be built in Adobe Flex. Why? Glad you asked. There are several reasons.

We’re most comfortable with it. The knowledge and experience are there. That means a quicker development time.
The Phidgets kit provides a nice Flex API. Which is, ya know, important.
We can make it an Air app. We can build a desktop app with Adobe Air, which is a “Good Thing”. Even better, we can still allow support for…
Game Development in Flash. There are quite a few Flash based games floating around on the internet. These games are designed for certain kind of user attitude, which lines up pretty directly with the user attitude we are assuming in the context of a public space. This attitude expects that games are quick to play and easy to understand. Even more importantly, we hope for our physical installation to stand as a long term development tool for the New Media students that follow us. We’d love for our work to provide opportunities for future students to learn about touch and gesture based interface design. The New Media program is pretty strongly focused on Flash as a development platform, so it is a good bet these future students will have a strong grasp of the environment.

The PINGplatform software will abstract the interactions with the Phidgets API, providing an easy interface to the data that a game developer will really care about. It will also provide tools to create common interface bits. What’s that mean? Think text input. Think list item selection. Finally, the platform will also provide an operating system of sorts, within which games will be chosen and loaded. This means lots of games can be played in the same physical installation.