Intangible Interactions Week 1
Questions
What is your definition of intangible interaction?

Normally, to interact with the digital world, we need some kind of explicit physical interaction. For example we would type on a keyboard mouse, or in more recent times, press on a touchscreen. What's common to all of these input methods is that they require the use of hands to do an action that wouldn't otherwise take place.

More recently, digital devices also have other input methods, specifically cameras and microphones, but most existing applications of these sensors requires them to be used in conjunction with a tangible interaction — for example, by tapping the shutter button in a camera app — to perform a meaningful action. Thus, these experiences are still gated on tangible interactions.

Intangible interactions are those that don't require tangible interactions to produce a meaningful action or result.

What is your favorite project that involves intangible interaction?

I have a hard time picking favorites, but the project that lives rent-free in my head is an experience is an installation where a mini soccer field was projected onto the floor, complete with a soccer ball. Passersby who walked onto the fake field could kick the fake soccer ball around, and the projected ball would move as if it were real.

I'm sure this was an art project at some point, but by the time I saw it, it seems like this kind of experience had already been fairly commified — I saw it in a shopping mall.

Have you seen any projects that trick your senses with sensory illusions?

I've seen quite a few projects that use sensory illusions to make you feel like you're manipulating the real world. I think the current hallmark of that is probably VR headsets, which present a complete, simulated visual field of view. My main gripe with devices and projects that use all-encompassing illusions is that it almost never looks or feels as good as the real thing, unless Apple-like levels of R&D have been poured into it.

I've seen a lot of very convincing illusions in everyday electronics. One example that comes to mind is noise cancellation — which generates more noise to cancel out existing noise. Another example: MacBook computers made after 2015 don't have have a trackpad button that physically clicks — a fine-tuned motor generates a quick vibration such that it feels like there's a physical button.

What is the relationship among intangible interaction, multi sensory experience and human perception?

Traditional interactive experiences include intentionally designed experiences, like bars and live concerts, which we have spent thousands of years iterating upon. Interactive experiences also include activities that are part of our daily routine, like walking to a destination or cooking, which our senses have evolved over hundreds of thousands of years to do well.

We make use of a wide variety of tangible and intangible interactions for all of these experiences, but when we use digital interactive experiences, the intangible interactions are extremely limited. To be able to enjoy digital interactions to the same extent, I think we need to create a similarly diverse palette of intangible interactions.

Project 1, Part 1
A device with intangible interaction

I picked a night-light with an automatic switch. The light turns on when motion is detected, and then turns off automatically approximately 30 seconds after motion was last detected.

How does it work?

From observation, there is a white, semi-translucent, rectangular window on the device, which is the sensor is likely behind. But what kind of sensor is it?

From doing some quick research, It looks like there are three common kinds of motion sensors:

  • Passive Infrared sensors, which detects a differential in infrared light between two slots. Humans emit a detectable amount of infrared light through black-body radiation due to our temperature.
  • Microwave sensors, which emit microwaves and measure the Doppler shift in the microwaves that are reflected back, and ultrasonic sensors, which emit sound and measure the time it takes for the sound to be reflected back.
  • Video cameras with software that detects significant differences between adjacent frames.

Which sensor is this device? The video camera possibility was easy to rule out, as the window is almost opaque and a using a video camera for this application does not make economic sense.

A further clue came from the the texture of the window:

It turns out that that is a Fresnel lens, which is a way to construct an otherwise-thick lens using significantly less material. Fresnel lenses are also used in lighthouses to focus light into a thin beam. Reading the Wikipedia article, I realized that I've seen Fresnel lenses also used in overhead projectors (only 90's kids will get it) — I've always wondered what the purpose was of those concentric ridges visible on the stage on the projector.

The Fresnel lens definitely narrowed down the motion-detection mechanism to a passive infrared sensor. The lens is needed to direct light from all directions into the sensor, which otherwise would have much narrower field of view.

I wasn't able to figure out how the sensors are embedded or programmed on the device. I'd assume that a microprocessor handles the inputs and is programmed to flip the light switch when a threshold is crossed. But does running a microprocessor even save energy compared to just leaving the LED light on 24/7? Turns out, very much so! A LED suitable for a night light uses around 500 mW, while a general-purpose Arduino uses only a few mW. The single-purpose microprocessor in this device probably uses even less.

This is a simple device: the only two errors this device can make are turning on when it shouldn't turn on, and not turning on when it should. I've been using it for a while, and I've noticed that it's very accurate on a technical level. When I'm moving in the vicinity of the sensor, the light will always activate, and when I'm in an adjacent room, I've never seen it turn on spontaneously.

However, this doesn't completely align with user expectations, specifically when the user is staying in the same room as the light. Reasonable user expectations would be either for the light to stay on if the user is in the same room, or for the light to stay off and only turn on when the user gets up. In practice, I've noticed that when I'm sitting or sleeping in the same room as the light, it will occasionally and arbitrarily turn on, resulting in a suboptimal experience. Because of this, this light is not suitable for bedrooms, only for hallways and rooms where where random light is not an issue.