Thinking about the Science of Communication and Interaction

Alien Senses

In the Galaxy Far Far Away…

What if sentient being evolved on a planet with permanent cloud cover? What if these being never saw stars? Would they still be able to discover the laws of nature?

These kinds of hypothetical thinking questions — the Gedankan Experiments, as Einstein put it — are very useful in science. I’ll try to use them here for analyzing product design and communication.

So what senses do we need to communicate? And what body appendages are necessary to produce this communication? Note that it helps keep track of these separately.


When I was little, I “designed” a language based on smell: each smell was assigned a character in an alphabet and, strung together in sequence, my smelly letters transcribed into words — the Aroma-bet. There were several problems with this:

  • It was difficult to get an alphabet-worth of distinct odors;
  • Arranged next to each other, the odors started to blend into each other, making “reading” difficult;
  • I got a very bad headache;
  • My mom didn’t like her expensive perfumes used in such a creative way…
  • And I couldn’t remember what letter each smell stood for, requiring the creation of a smell-o-dictionary, which in turn used a lot more of my mom’s perfume, which led to more trouble…

But other then that, I loved the idea.

I haven’t thought back on my smelly language in years. But it came to mind again when I was trying to work out the boundaries of communication problem: what are all the possible ways we can communicate with our senses given the environmental and social limitations?


We use sound to communicate — we’ve developed sound-based language, where words can be composed of phonemes which are stung together to make thousands of possible combinations of sounds to represent concrete and abstract ideas. To do so, we need two things: ability to generate sounds (just the right kinds of sounds) and the ability to receive and comprehend sounds.

There are many different ways sounds can be made. Humans use only a subset of what’s possible. We tend to stick with what we can produce orally: phonemes, clicks, whistles, pops, and the like. And modern languages limit this range to mostly phonemes — ancient click and whistle languages are mostly gone now.

But other alien civilizations can use sound to communicate in different ways. Multiple body parts can be involved: clapping, stomping, whistling (ala whales and dolphins). And the acoustic range could be wider or narrower or simply different. Different environments allow for better transmission of certain frequencies.

And of course to use sound for communication, there needs to be a receiving organ. Humans need ears to register sound and audio-processing centers of the brain in order to comprehend the information streaming in this format. Some individuals with audio-processing disorders have problems processing sounds fast enough to understand them, unable to tell the difference between “b” and “d” — turning “dad” into “bad”.

Advantages of Sounds Communication:

  • don’t have to be touching or facing the each other to communicate;
  • there’s a limited range of communication — individuals around the globe don’t have to be participants in every conversation;
  • can use and intermediary device to carry sounds to extend the range of communication — think phone, radio, TV, the Internet;
  • can use different materials to change the speed of transmission — sound travels at different speed in different mediums;
  • communication can be conducted in total darkness.

Disadvantages of Sounds Communication:

  • limited range — have to be within certain distance to hear;
  • sound is easily barred by walls and doesn’t do corners well;
  • communication is serial — sounds are strung one after another in time, requiring a certain amount of time to give and receive a message;
  • privacy is easily compromised;
  • multiple streams of audio information are very difficult to follow — common problem in a noisy environment.


We also have visual communication — we use visual symbols as phonemes. We can string together an infinite combination of letters and words to express ideas.

But an alphabet is not the only method of visual communication. We can use images, as in photos, graphs, movies, cartoons, illustrations, symbols, logos, etc.

We also use visual information to gather data on emotions — during face-to-face conversations, the emotions expressed on the face and those projected by body motions and stands add an enormous amount of additional context to the spoken words. This is why telephone conversations and radio plays require more concentration from the listeners — there’s only the audio stream of information, which is very limiting.

semaphore Beatles album cover Visual communication can also be serial without being written down. Sign language is an example of using gestures to communicate visually. Semaphore is another system for communication at a distance by means of visual signals with hand-held flags, rods, paddles, or other devices.

John Cleese Traffic Lights And of course we have light signals that are used for directing vehicles: red for stop, green for go. This is also an example of visual communication.

There are many possibilities for visual communication.

Advantages of Visual Communication:

  • the receiver of information can get the whole message at once — while books require time to read and the reader doesn’t get to “see” the whole story at once; illustrations allow for the whole to be viewed at once (even if it takes time to carefully examine the visual message);
  • can superimpose multiple messages within different frequencies;
  • allows for interesting comparison of data when presented in non-serial way;
  • can be transmitted via different media: TV, computers, paintings, photographs, etc.;
  • can be used to communicate at a longer range than audio (consider lighthouses);
  • light travels through vacuum.

Disadvantages of Visual Communication:

  • requires a source of illumination — can’t read in the dark — either external or internal (think backlit);
  • privacy is easily compromised;
  • has to be in the direct line of sight of the message — visual information does corners even more poorly than audio messages;
  • data can be easily missed (that’s why eye-witnesses are so unreliable);
  • can be cloaked by poor atmospheric conditions like fog or smog;
  • the receiver of communication requires skills to understand the transmission of information (true for all forms of communications).


Haptic are all other human-possible forms of information transmission: body movements (kinetic), facial expressions, hand signals, pheromones, smell, surface changes (touch). Aside for pheromones and smell, most of these require that the receiver (not the sender) of information is capable of getting visual signals.

But there’s one more sense that doesn’t require sight or hearing — touch! The most common example of touch communication is brail — a system of raised bumps on the surface that allows a person to “read” the message like one would read an alphabet. Only the reader of brail can use finger tips to gather more data at once than a visual reader of the alphabet by moving both hands from sides to the center. The reading of brail is still serial in nature, but more two dimensional than sight reading.

Helen Keller and Anne Sullivan 1898 There’s also a form of sign language that is communicated through touch — Tactile Signing — think of the story of Anne Sullivan and Helen Keller.

There are advantages and disadvantages of haptic communication, some already explored above in the Visual and Audio communication paragraphs. The most serious disadvantage of tactile communication is the need for proximity between the conversationalists. But what are the advantages of this mode of informational transfer?

  • can converse in the dark and in noisy environments;
  • there’s no mistaking who the message is intended for — the people are in direct contact with each other during data transmission;
  • can use more than just changes in the surface texture to communicate: electrical charges, temperature variations; weight differences; shapes and grooves within the materials; orientation in space; and even sound can all be felt through touch;
  • for dyslexics, the serial nature of tactile communication can provide support and reduce error transmission.

Inca Khipu knot language Touch is a very rich medium of communication. Inca Quipu/Khipu culture used a complicated set of knots as a system to record information — the “reader” felt-up a string with knots and extracted data from the recorded message. Clearly, there are many interesting ways that touch can be used to conway information.

Back to Aliens & Gedankan Experiments

Comb jellyfish bioluminescence The range of possibilities for sensory-based communication is limited by the physical laws — telepathy is great in sci-fi novels, but is not possible in real life (although, direct brain stimulation as a way of transmitting data is possible). And the options of which senses are most useful for communication are depended on the environmental conditions under which communication takes place. Alien life living deep in the ocean would have issues with visual communication without internal light sources — thus we have wonderful bioluminescence.

Looking to our own oceans again, we can see the use of sound, electricity, and the slight variations in the “taste” of water to extract data from the environment and to communicate. What else? What other variations in the environment that are easily generated and consistently duplicated can be appropriated for communication?

These Gedankan Experiments are not simple flights of fancy. Think about possibilities, freed from excepted norms, can help product designers find novels ways to communicate. Gestural interface has its roots in sign language. What can smell and taste do for us and our devices? How can we use easily generated and consistently duplicated variations to conway a message? And would this message expand the range of possibilities for device uses?

  2 comments for “Thinking about the Science of Communication and Interaction

  1. December 5, 2011 at 3:08 pm

    My son would call this “clumping” — a phenomenon where ideas tend to clump in time. This week, “Wake Up World” — an online science magazine — published a story on the discovery of “sono-pictorial” language — a way for dolphins to communicate with each other! This seems to be a real language, with nouns and verbs and ways to express ideas that were previously never spoken. And all of this by intelligence other than our own! Dr. Jack Kassewitz of described how dolphins were using echolocation to “scan” and image and then reproduce it at will to communicate information to other dolphins at other times. The researchers were able to record an son-pictogram of an object and have a dolphin who never saw that object before identify it with 80% accuracy! Amazing! Here’s a complete article with links to research:

    And a day later, November 29th, Scientific American published an article on Raven’s ability to use gestures to communicate — a first among non-primates:

  2. December 8, 2011 at 7:39 pm

    Here’s my response from Human Factors and Ergonomics Society group on LinkedIn (please follow our discussion there).

    My mother-in-law developed a machine: Sensorama. She collected the smells and sounds of New York City in addition to video. The machine allowed the user to sit on a motorcycle-like device and see in all directions, listen to street sounds, and smell traffic (this was before they figured out that smelling car exhaust was probably not good for you). It was an interesting and more sensory rich way of exploring the city than Google maps… That was several decades ago.

    We still have a fairly narrow sensory interaction with our devices–we record a very limited set of sensory information.

    And we clearly have to keep processing of sensory information from perceiving such information. While my son might be hard of hearing, he has amazing musical processing abilities and is a very good musician.

    What I was trying to do with the gedankan experiments on my blog was to explore the ways we can use this “external” data to improve interaction experience for users. We are finally starting to use gestures (a bit like tactile signing), but we are not using the full spectrum of what’s available to us. Some is clearly impractical, but imagining the axis of time and space necessary in communicating information, perhaps we can develop better interaction and decrease user error.

    And thank you very much for your comment and forgive my tardy response–the Thanksgiving family togetherness brought a bit of pestilence into our household…

Comments are closed.