Touch2Know Dexterous In-Hand Liquid Sensing via Mutual-Capacitance Fingerprints

Robots and people often need to know what liquid is inside a sealed bottle before acting, but vision can fail for opaque or visually similar contents, and continuous cameras may be undesirable in shared spaces. We present Touch2Know, a dexterous-hand sensing system that identifies liquid type and container type through the container wall during normal in-hand grasping, without opening the bottle. Touch2Know integrates five low-profile flex-PCB electrodes with guarded and shielded routing on a multi-finger hand, and measures multi-channel mutual capacitance to reduce common-mode drift and grounding sensitivity. An event-driven adaptive grasp controller stabilizes contact and limits over-compression across PET and glass bottles. For learning, we propose CapTF (Capacitive Temporal Fusion), a CNN–Transformer model that jointly predicts (i) liquid class and (ii) container type from 4-channel capacitive time series. We collect a dataset spanning 19 liquid classes (11 household liquids plus salt-water and sugar-water concentration series) and 4 container types over four days. The task is fine-grained, including visually ambiguous cases and liquids with closely related dielectric/conductive responses through container walls. With per-trial baseline-subtracted signals, CapTF achieves 95.07% / 99.34% liquid/container accuracy under a balanced split and 90.00% / 98.16% on the Day-4 holdout test, outperforming LSTM, XGBoost, and a vanilla Transformer in liquid classification.

Grasp → sense through the wall → recognize

Opaque bottles hide their contents from cameras; shaking, tilting or opening adds extra actions; and most non-visual approaches need specialized end-effector hardware. Touch2Know turns the grasp itself into the sensor. A multi-finger enveloping grasp stabilizes containers of different diameters and materials, keeps distributed contact around the bottle, and provides several sensing viewpoints at once, so the measurement is less sensitive to small placement errors and matches how a robot really handles a bottle.

Figure 1. Touch2Know overview. A UR5-mounted dexterous hand performs an enveloping grasp and records 4-channel mutual-capacitance signals through sealed containers using five on-hand flex-PCB electrodes. The dataset covers 19 liquid classes and four container types over four days.

Guarded, shielded flex-PCB electrodes on a dexterous hand

Electrodes are fabricated on an 18 µm Cu / 25 µm PI / 18 µm Cu laminate. The object-facing side carries a rectangular sensing pad enclosed by a 1 mm GND guard ring; the back side routes the signal trace inside a grounded copper shield. A coax-like SMA launch and short RF coax runs keep a continuous shielding path to the readout board, and the detachable SMA pair makes any electrode easy to replace. Each electrode is mounted at the second finger joint on 1 mm foam tape, so it sits inside the enveloping contact without interfering with the fingertip 3-axis force sensors.

Figure 2. Flex-PCB electrode workflow and system integration for multi-channel mutual-capacitance sensing, from material stack-up (a) to on-hand integration (g) and readout (h).

CapTF: Capacitive Temporal Fusion

A single grasp yields a 4-channel sequence that passes through distinct phases: a short precontact window, a variable-length closing transient, and a long steady-state hold. CapTF is built around that structure. A multi-scale 1-D CNN front-end (kernels 7 → 5 → 3, with a residual block) resolves local contact dynamics; a pre-LN Transformer encoder with a learnable positional encoding captures long-range phase-to-phase context; and mean, max and attention pooling are concatenated into a shared representation.

Because the measured coupling depends jointly on the liquid and the container wall, CapTF is trained as a multi-task model with two heads, one for liquid class and one for container type. Predicting the container encourages the shared backbone to learn features that are less entangled with container-dependent offsets, which improves liquid separability by roughly 3 pp over a single-task variant.

Figure 3. CapTF architecture. A multi-scale 1-D CNN embeds the 4-channel mutual-capacitance input; a Transformer encoder with learnable positional encoding captures temporal dependencies; multi-head pooling (mean + max + attention) produces a shared representation fed to two task-specific heads.

Fine-grained liquid recognition through sealed bottles

We evaluate on 19 liquids (water, ethanol, grape juice, milk, milkshake, oil, soy sauce, syrup, vinegar, handwash, dishwashing liquid, plus salt water at 2/4/6/8% and sugar water at 5/10/15/20%) in four containers (hard and soft PET, thin and thick glass), under two splits: a stratified 70/30 balanced split and a Day-4 holdout split that trains on Days 1–3 and tests on Day 4. Splits are at the trial-file level to prevent leakage. Delta denotes per-trial baseline-subtracted features; Raw uses the unprocessed signal.

Figure 4. CapTF with delta features: (A) liquid confusion matrix, balanced split; (B) liquid confusion matrix, Day-4 holdout; (C, D) container confusion matrices; (E) t-SNE of pooled embeddings for the 19 liquid classes.

Limitations and next steps

The current study uses four bottle types filled near-full (450 mL). Future work will cover broader container geometries, fill-level variation, temperature effects, calibration transfer, open-set recognition, direct prediction of liquid relative permittivity, and integration with downstream manipulation such as pouring and handover.

Poster

BibTeX

@inproceedings{deng2026touch2know,
  title     = {Touch2Know: Dexterous In-Hand Liquid Sensing via
               Mutual-Capacitance Fingerprints},
  author    = {Deng, Ruixiang and Shangguan, Zhegong and Hu, Yang and
               Bai, Haozheng and Li, Tingcheng and Cangelosi, Angelo
               and Yang, Wuqiang},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots
               and Systems (IROS)},
  year      = {2026}
}