PitchIn: Eavesdropping via intelligible speech reconstruction using non-acoustic sensor fusion

Jun Han, Albert Jin Chung, Patrick Tague

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Citations (Scopus)

Abstract

Despite the advent of numerous Internet-of-Things (IoT) applications, recent research demonstrates potential side-channel vulnerabilities exploiting sensors which are used for event and environment monitoring. In this paper, we propose a new side-channel atack, where a network of distributed non-acoustic sensors can be exploited by an atacker to launch an eavesdropping atack by reconstructing intelligible speech signals. Specifically, we present PitchIn to demonstrate the feasibility of speech reconstruction from non-acoustic sensor data collected offline across networked devices. Unlike speech reconstruction which requires a high sampling frequency (e.g., > 5 KHz), typical applications using non-acoustic sensors do not rely on richly sampled data, presenting a challenge to the speech reconstruction atack. Hence, PitchIn leverages a distributed form of Time Interleaved Analog-Digital-Conversion (TIADC) to approximate a high sampling frequency, while maintaining low per-node sampling frequency. We demonstrate how distributed TI-ADC can be used to achieve intelligibility by processing an interleaved signal composed of different sensors across networked devices. We implement PitchIn and evaluate reconstructed speech signal intelligibility via user studies. PitchIn has word recognition accuracy as high as 79%. Though some additional work is required to improve accuracy, our results suggest that eavesdropping using a fusion of non-acoustic sensors is a real and practical threat.

Original languageEnglish
Title of host publicationProceedings - 2017 16th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2017
PublisherAssociation for Computing Machinery, Inc
Pages181-192
Number of pages12
ISBN (Electronic)9781450348904
DOIs
Publication statusPublished - 2017 Apr 18
Event16th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2017 - Pittsburgh, United States
Duration: 2017 Apr 182017 Apr 20

Publication series

NameProceedings - 2017 16th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2017

Conference

Conference16th ACM/IEEE International Conference on Information Processing in Sensor Networks, IPSN 2017
Country/TerritoryUnited States
CityPittsburgh
Period17/4/1817/4/20

Bibliographical note

Funding Information:
We would like to thank our shepherd, Rui Tan, and anonymous reviewers for their valuable comments. This research was supported in part by the National Science Foundation under grant CNS-1645759. The views and conclusions contained here are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either express or implied, of CMU, NSF, or the U.S. Government or any of its agencies.

Publisher Copyright:
© 2017 ACM.

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Information Systems
  • Signal Processing

Fingerprint

Dive into the research topics of 'PitchIn: Eavesdropping via intelligible speech reconstruction using non-acoustic sensor fusion'. Together they form a unique fingerprint.

Cite this