Gesture as Inclusive Interface#Gesture #Interface #ComputerVision #GestureRecognition #MachineLearning #HumanComputerInteraction #SpeculativeDesign #TouchDesigner #MediaPipe #TeachableMachine #Culture #Diversity
This work began with a question on the fundamental limitations that standardized interface functions impose on interaction systems. In particular, gesture-based interface enables touchless, natural interactions through various movements of body parts, yet they still rely on traditional touchscreen input methods such as tap, pinch, and swipe. It creates barriers for users with disabilities and often fails to account for culturally diverse hand gestures. In Gesture, Vilém Flusser, a Czech philosopher and writer, argues that gestures are a mediating process between apparatus and human, carrying cultural and ideological meanings. He emphasizes that gestures reveal our behaviors, perceptions, and relationship to the world. As Interactive design increasingly influences how we view ourselves and interact with the world, it is essential to unlock its full capacity to respond ethically and empathically to users’ diverse identities and cultural backgrounds. Building on this idea, my project re-imagine the current gesture-based interface by embedding diverse cultures, values, and worldviews into a more immersive and accessible form of HCI.
Project Objectives
This speculative project explores a range of finger, hand, and body gestures. It demonstrates how customized gestures can replace and perform existing navigational, action, and transformational functions by harnessing pose and gesture recognition through camera vision.
1. Diversity
The gesture interface should consider the cultural meanings of various body movements.2. Inclusivity
The gesture interface should consider users with varying physical abilities and dexterity.
3. Functionality
The gesture controls should perform the three primary actions: navigation, zoom, and select.4. Feedback
The gesture interface should provide clear visual feedback to users with sensory impairments.
Process & Methods
Culturally Diverse Hand Gestures
Hand Gestures of the Legendary Peking Opera Master Mei LanfangHand Gestures of the Legendary Peking Opera Master Mei LanfangThe four basic hand gestures of Khmer dance, and a specific mimed spplicationUkiyo-E by Utagawa KunisadaAsanas are the physical postures or poses in yoga“Pantea”, a sacred symbol of the god SabaziusHand-signs of Bushmen warriors and huntersTai Chi BasicsRoman Hand of SabaziosExercises for the chest and trunkUday Shankar, a renowned Indian dancerSign LanguageTaoist Qigong postureDoris D. AerobicsHow Victorian Women Managed To Exercise While Wearing CorsetsHow to Properly handle LPs
Hand Gesture Explorations
Body Gesture Explorations
Hand Gesture Controls Demo
Navigational Gesture: ScrollTransform Gesture: RotateAction Gesture: Tap