Hand gesture classification utilizing high-quality structured knowledge reminiscent of movies, im-
ages, and hand skeletons is a well-explored drawback in pc imaginative and prescient. Alterna-
tively, leveraging low-power, cost-effective bio-signals, e.g., floor electromyo-
graphy (sEMG), permits for steady gesture prediction on wearable gadgets.
On this work, we purpose to boost EMG illustration high quality by aligning it with
embeddings obtained from structured, high-quality modalities that present richer
semantic steering, in the end enabling zero-shot gesture generalization. Specif-
ically, we suggest EMBridge, a cross-modal illustration studying framework
that bridges the modality hole between EMG and pose. EMBridge learns high-
high quality EMG representations by introducing a Querying Transformer (Q-Former),
a masked pose reconstruction loss, and a community-aware mushy contrastive learn-
ing goal that aligns the relative geometry of the embedding areas. We eval-
uate EMBridge on each in-distribution and unseen gesture classification duties and
show constant efficiency positive factors over all baselines. To the perfect of our
information, EMBridge is the primary cross-modal illustration studying framework
to attain zero-shot gesture classification from wearable EMG alerts, displaying
potential towards real-world gesture recognition on wearable gadgets.
- † College of Southern California
- ** Work carried out whereas at Apple
