Class 17, Reading 2

Reading

• Intimate Interfaces in Action: Assessing the Usability and Subtlety of EMG-based Mottionless Gestures
  
• Costanza, et al.
  

Outline

• Introduction
  
• Background
  
• System Description
  
• Evaluation
  
  • Experiment 1: Controlling an Audio Menu
    
  • Audio Menu
    
  • Experiment 2: Noticeability of Subtle Gestures
    
• Conclusion and Future Work
  

Notes

• Introduction
  
  • Nature of mobile communications devices
    
    • Often personal and private, but takes place in public spaces
      
    • Contrast generated by private communication in public problematic for:
      
      • Confidentiality of person who communicates
        
      • People who are disrupted and forced to listen to stranger's private conversation
        
    • Mobile phone users generally play active role; can decide whether to:
      
      • Engage in phone conversation in public, or
        
      • Wait until they are in protected, isolated area
        
    • Bystanders have passive role
      
  • Nature of mobile communication
    
    • Often phatic, whose only function is to perform a social task, as opposed to conveying information
      
    • Personal communication through mobile devices is a largely minimal
      
      • Involves exchanges of little information
        
      • Real function is to provide
        
        • remote awareness
          
        • reassurance to oneself about possibility to connect to one's own social network
          
      • Design of interfaces and interaction techniques need to take into account above observations
        
  • Proposal: mobile interfaces that provide affordance for minimal interaction
    
    • Users should exchange simple signs of presence with remote partner or friend in an easy, natural, and inconspicuous manner
      
    • Mobile interfaces should offer users some simple, low-bandwidth, low-attentional operations that can be performed without disruption or embarrassment even in public or social context
      
  • Attentional demands have been observed before mobile technology
    
    • Main involvement
      
      • Absorbs the major part of an individual's attention and interest
        
      • vVsibly forming the principle current determinant of his actions
        
    • Side involvement
      
      • Activity that individual can carry on in abstracted fashion
        
      • Doesn't threaten or confuse simultaneous maintenance of main involvement
        
  • Previous work: motionless gestures for subtle input as step towards defining intimate interfaces
    
    • Intimate interfaces – mobile interfaces that are discrete and unobtrusive
      
    • Previous work was wearable controller that takes advantage of EMG signal to sense isometric muscular activity
      
    • Isometric muscular activity – voluntary muscle contractions results in little or movement
      
    • Initial study:
      
      • Perform subtle gesture each time they were prompted through audio cue
        
      • Result:
        
        • Gestures are easy for people to learn
          
        • Can be successfully recognized with simple algorithm
          
  • Paper's research
    
    • EMG-based subtle motionless gestures by examining
      
      • 1. The usability of such gestures in realistic mobile applications
        
      • 2. How noticeable they are to informed observers
        
    • Presents an improved wearable armband device made invisible by wearing it under clothing; includes
      
      • Gesture sensor
        
      • Tactile display
        
• Background
  
  • Biosignals
    
    • Defined as electrical signals generated by physiological activity
      
    • Sensed with amplifiers, which can be low-cost embedded, battery-operated devices
      
    • Can be used in broader range of applications in HCI
      
  • EMG signal
    
    • Defined as an electrical voltage signal generated by muscle activity
      
    • With EMG, possible to sense isometric muscular activity (i.e., muscular activity that does not produce movement)
      
    • Paper's authors and EMG
      
      • Proposed motionless gestures based on isometric activity
        
      • Developed wearable system and algorithm to detect gestures while users are mobile
        
    • Social acceptance of mobile interactions
      
      • Novel interfaces proposed to address this factor
        
      • No explicit evaluation in previous work in terms of visibility or noticeability by others
        
• System Description
  
  • Intimate Communication Armband
    
    • Conceived as a generic input/output peripheral for mobile devices
      
    • Worn on the upper arm, invisibly under clothes
      
    • Sense explicit subtle gestures and provides tactile output
      
    • Connects wirelessly via Bluetooth to phone or PDA, sitting in user's pocket or bag
      
    • Emits signals every time gesture is recognized, accepts signals to activate tactile display
      
    • Mapping strategy from signals to specific interface actions and events left open to application designer
      
• Evaluation
  
  • Experiment 1: Controlling an Audio Menu
    
    • Examined the usability of the EMG sensor within a realistic interaction scenario in a mobile context
      
      • Subjects simultaneously:
        
        • Selected items from an audio menu through subtle gestures
          
        • Engaged in a simulated mobile task
          
      • Two conditions compared:
        
        • 1. Two armband controllers were used at the same time on the two arms
          
        • 2. A single controller was used to select one of multiple options presented over time
          
      • During experiment, tactile output from armband device was used to provide feedback about gesture being recognized
        
    • Experiment tested the follow hypotheses:
      
      • 1. EMG-based recognition of subtle gestures can be used for input within multimodal interfaces
        
      • 2. Interface bandwidth can be increased by using single controller to select one of multiple choices presented over time
        
      • 3. EMG controllers can be used concurrently on multiple muscles to increase interface bandwidth
        
      • 4. Using multiple muscles is more efficient than using a single muscle, because of reduced time pressure
        
    • Experimental Design
      
      • Walking
        
        • Using the wireless EMG device, participants performed three walking tasks:
          
          • Without controller any interface (to determine the subject's preferred walking speed)
            
          • While controller the audio menu with one arm
            
          • While controlling the audio menu with two arms
            
        • Mobile context of walking: to take measurements of usage of device whilst
          
          • Users were mobile
            
          • Not as formally controlled as laboratory study, which lacks realism and ecological validity
            
      • Walking speed of subjects
        
        • Used as an index for interface's effectiveness
          
        • If mobile interface has negative effect on users, it's reflected in users slowing down while walking
          
        • Subjects' preferred walking speed measured at beginning of experiment as reference for comparison
          
      • Participants:
        
        • Given written instructions that study was assessing EMG-based interface
          
        • Would control system using their biceps while walking using subtle contraction with arm relaxed and on their side
          
        • Were encouraged to perform contraction in subtle way without attracting attention
          
      • Familiarization session
        
        • Participants stood and received tactile feedback when system recognized contraction
          
        • Lasted 10 minutes, during which participants performed gestures while walking around obstacles
          
        • First performed on arm with dominant hand, then subsequently on other arm
          
  • Audio Menu
    
    • Audio menu simulates what could be employed on mobile phone
      
      • 1. "Reject call"
        
      • 2. "Ignore call"
        
      • 3. "Reject call and send, 'I will call you back later.'"
        
      • 4. "Reject call and send, 'In a meeting; call back if urgent.'"
        
    • Two conditions to access menu:
      
      • 1. Two-arm - Subjects used arm of dominant hand to sequentially advance the menu (next action), and other arm to select current item
        
      • 2. One-arm – Current item was automatically advanced, two seconds after item description was read
        
    • To simulate realistic conditions, subjects promoted with audio stimuli mimicking incoming calls
      
    • Results
      
      • Subjects performed correct selections of items from audio menu 96.2% of the time
        
        • Two-arms condition: 97.6%
          
        • One-arms condition: 94.6%
          
      • One-way ANOVA showed no significant differences in subjects' walking speeds for one-arm and two-arms condition
        
    • Discussion
      
      • Users were able to control audio menu consistently while mobile
        
      • 1^st hypothesis: High overall accuracy indicates EMG can be used successfully in complex and multimodal interfaces
        
      • 2^nd and 3^rd hypothesis: Subjects performed with high accuracy for both conditions
        
  • Experiment 2: Noticeability of Subtle Gestures
    
    • About
      
      • Principal motivation for EMG-based interfaces in the context of mobile HCI is subtlety
        
      • People around user should not notice and should not be disrupted by his or her interaction with a mobile device
        
      • Second experiment designed to measure how noticeable subtle gestures are based on two hypotheses:
        
        • 1. Subtle gestures detected through EMG are generally not noticeable
          
        • 2. The gestures are particularly unnoticeable when the arm-band device is hidden under clothing
          
    • Experimental Design
      
      • Subjects:
        
        • Asked to watch video recording of trained user activating armband
          
        • Try and guess when interface was being activated
          
      • Video shows actor in front of neutral background
        
        • Subtle gestures performed with upper right arm while talking with someone off screen
          
        • No audio
          
      • Instructions to subjects:
        
        • During video, five buttons below sentence "The person in the video is activating the EMG interface":
          
          • Definitely
            
          • Very probably
            
          • Probably
            
          • Possibly
            
          • Not sure
            
    • Description
      
      • Same subjects taking part in previous experiment also asked to participate in video rating
        
    • Results
      
      • Guesses were correct if they were up to 1.5 seconds apart from groundtruth
        
      • Overall correct guesses:
        
        • Long sleeves: 13.9%
          
        • Short sleeves: 33.1%
          
        • Close-up: 75.9%
          
    • Discussion
      
      • Hypothesis 1: results show that subjects cannot easily guess when interface is activated, confirming interface is subtle
        
      • Hypothesis 2: when armband is hidden under clothing, it becomes less noticeable
        

Comments

• In the introduction, the paper states:
  
  • "…mobile interfaces should offer users some simple, low-bandwidth, low-attentional operations that can be performed without disruption or embarrassment even when one is in a public or social context, …"
    
• The paper addresses this by examining subtle motionless gestures. While subtle motionless gestures are one solution to the envisioned mobile interfaces, it's one of several ways to accomplish this. In fact, a paper written in CHI 2010 by Rico, et al. examines socially acceptable gestures that are not subtle or motionless at all, yet still doesn't violate the criteria of the envisioned mobile interfaces.
  
• In the second experiment of noticeability of subtle gestures, the authors write "People around the user should not notice and should not be disrupted by his or her interaction with a mobile device". The paper then goes on and focuses on hypotheses for the important of subtle gestures, but the Rico paper believes that this can be accomplished through socially acceptable gestures that aren't necessarily subtle. For example, people text message silently or using multi-touch gestures on their phone is both socially acceptable and not distracting, but they are far from subtle.