Breakthrough Ideas: Face as controller.

I remember when I first saw the E3 presentation, I was so impressed with the idea of using human body as controller. Years later, I thought to myself what about using human face as a controller to control viewing. Therefore I did an undergraduate research in trying to use face tracking data to control viewing panoramic images in real time. Because of the instability of the face tracker that was used in my research, the result wasn’t really promising. So I didn’t end up publishing any paper on this. Then in the same year, I came across FaceAPI, which is the link down below. They basically did the same thing but with much more success than I did.

I think incorporating face tracking data into controlling computers can bring so many changes to interface design. We can adjust the images shown on computer monitor according to the head movements of the users, which would allow more information to be shown without changing the size of the screen (i.e. when the user look left, we can show one the left part of the desktop, and when the user look right we can just show the right part of the desktop).  The amazing part of this is that this doesn’t not even require special hardware. Both in my research and FaceAPI, the only additional equipment needed is a web camera, which is already available in most laptop computers and computer monitors.

However, the big challenge is calibration of control. The dilemma is that small movements of the face are hard to track and large movements are counter-intutive given that the user is looking at a screen close to him or her. Given the limitation of  accuracy and robustness of current face tracking algorithm, reaching the perfect calibration of control is almost impossible. Because we’re dealing with interaction here, a tiny bit of delay or mis mach would cause a lot of confusion for users. I think we still need to be cautious in experimenting with this. But I hope that there will be future advances in this area.



Thoughts on the CGT 512 Human Factors of Computer Interface Design

As the semester is coming to an end, now is a great time to exam how I benefit from this course and how can changes benefit future students studying this course.

This course is heavily focused on learning through assignments. The lectures for the most part is either an open discussion of the readings, or a small group in-class group project. I absolutely love having class this way, because I believe at the end of the day it’s not much about what we know, but what we can do with what we know. So in the following paragraphs, I divide in to different kinds of assignments and talk about how I feel about them.

First let’s talk about Research Article Analysis (RAA). Now it is very clear to me that as graduate students, we must be able to read research articles. That’s why courses like CGT 501 and CGT 511 are drilling us to do all the time. However, I don’t see how research articles play an important role in this course. I’m always having trouble finding research articles that I’m interested in. It’s really hard for me to find articles that are closely related to the materials that we learn in this course. I can think of two reasons for that. One is that the methods that we learned in this course are already established years ago, and it’s hard to find articles that are that old. The second one is that the methods and principles that we learned from this course are rarely originated from scholarly research. They are mostly based on experiences and experiments of UX practitioners such as authors of our text books. I managed to find 2 that are related to UR 4 about post-test questionnaires, and that’s it. And 5 total articles are a lot. I would like to see changes on this in the future.


Next let’s talk about Usability Reports (UR). I think having people forming teams doing this is great. It’s a great opportunity to learn not only from the assignments but also our classmates. But I really don’t understand all the intervention that Dr.V put into forming teams. First is the restriction of working with people that you haven’t worked with. And from UR 3 and 4, teams are assigned by Dr.V. I think building dynamics of a team takes time for everyone in the team to buy in. And only when you already get a strong and stable team, can you handle this amount of pressure coming from assignments like UR 3 and 4. We see teams in reality shows collapse all the time, because they are a bunch of strangers randomly put together, and everyone is for themselves. All this fighting and drama in reality shows are not because these people are all bad in nature or the show is scripted. It’s because a lot of things are on the line, and people just have a hard time trusting other people. I think giving people the freedom to choose for themselves definitely helps with this. And I would also recommend especially for UR 3 and UR 4 to have people pitching their ideas in front the class, and have other people choose which team they want to join. I think it’s really really really helpful for people to have something that they all agree on before they start working together. It’s my nightmare to come to a team meeting to have 4 people with 4 drastically different ideas. And my nightmare just gets even worse, when you see one person with multiple drastically different ideas. For people who have worked with me, you may know that I always work more comfortably with my idea. It’s not a matter of “mine is always the best”, but style. I don’t think I’m the only case in this matter. For example, although the groups for the two presentations were changed, you see some styles remained. Some people may be offended by other people’s style, so I think it’s important for people to be able to choose whether they want to be part of it or not.

Blog assignments are a great way of provoking thinking of the things that we’ve learned from this course. I love the idea of having the instructor hand pick the best of the week so we won’t need to follow through all the blogs. But I think there are too many of them. It’s so difficult to have an opinion or an interesting thought on every single thing. And what Dr. V described as “engaging the materials” is just me being lazy. It’s much easier to lead people believe that you’ve put thoughts into something when you opt to argue against it. On the other hand, you don’t have much to say besides repeating and summarizing things if you choose to agree with it. I heard some people get points off for doing that. I feel terribly guilty for not suffering the same punishment. However, if we all deserve to be punished, then it can only be that there is something with the rules. I’ve been following some of the blogs because of this course. And I don’t see them, posting in the speed that we do in our course. If we spread out the posts, it’s like two posts per week. This is kind of insane.


Thoughts on Microsoft’s so called futuristic video

Many of you may remember that one of the speaker from Microsoft mentioned in his talk about a futuristic video of what Microsoft has accomplished. I believe I’ve found the video here.

A lot of you may know that I’m not a huge fan of Microsoft with XBOX being the only exception. I’ve got nothing against them. I’ve been Windows user from since Windows 95. My conversion to apple started when I bought my first iPod. I think the problem with Microsoft is not that they don’t make good product. It’s their vision of the “future” is not as far as user’s expectation, and other industry pioneers.

Let’s take this video for an example. One may argue that there are indeed many cool features in this video. But what I get from this video is that Microsoft imagines the future to be restricted to 2D interface, just like our PCs and tablets today. This is not a surprising at all. Man kind has suffered from this restriction since the first cave painting came out. Because of this, our 3D thinking and imagination have not been developed as good as 2D thinking and imagination. We are able to perceive the world in 3D, but the virtual world that we created in books, paintings, magazines, movies, TV shows and games only exist in 2D. I think now it’s time to fix this.

3D interface does exist in the area of augmented reality. The idea of this is using integration of real world and virtual world to provide more information to users. There are several practices of this technology that exist today, mostly for military application. For example tank drivers use this technology to measure the target area of attack, and fighter pilot use it to track enemy aircraft.

The obvious advantage of augmented reality to the 2D design that’s shown in this video is that we don’t need to turn everything into to a display. We only need one display per eye, two per person. We will not only be able to accomplish what’s shown in this video with a much cheaper cost, but also do more in  3D. Imagine you are an architect testing out the design of your building and you are standing at the location for your new building. With this technology, you will be able to integrate the 1:1 scale virtual model into the location to see the exact result. It’s way cooler than touching your fridge to check the weather.

However, there is one thing that’s stoping augmented reality to become a “thing”. That’s the issue of display. Intuitively, you want to get the display as close to your eye as possible. Some people are working on making display on contact lenses, and some are working on projecting images directly to the retina. There aren’t any huge progress on these yet. Making very thin touch screen display however is much more promising. This is probably the reason why Microsoft opts to choose this.

Although design-wise I think 3D interface with augmented reality I think is much better than what’s shown in this video, in the end it all comes to whichever that comes first and whichever is cheaper.