NeTS: Small: Reliable Task Offloading in Mobile Autonomous Systems Through Semantic MU-MIMO Control

Synopsis

Mobile autonomous systems (MASs) such as self-driving vehicles and drones have a pivotal role in critical applications such as urban mobility, precision agriculture and remote surveillance. To achieve their tasks, MASs increasingly rely on high-throughput low-latency streaming of computer vision tasks (e.g., object detection) to edge servers. However, ephemeral environmental factors such as blockages, congestion and fading may erratically interrupt the flow of tasks to the edge servers. Existing work has addressed computation and communication issues of task offloading by MASs separately, which necessarily leads to suboptimal solutions. Task accuracy, indeed, is inevitably tied to the quality of the multimedia data being sent to the edge, which in turns depends on the adopted wireless strategy. However, the wireless parameters being used depend on the quality of data being sent (the more compression, the higher the latency), which ultimately impacts the desired task accuracy. Thus, to achieve applications that are “resilient-by-design” without compromising task accuracy, the semantics of the multimedia data must be holistically and fundamentally intertwined with real-time optimization of wireless transmissions. The core advance of this project is the design and experimental evaluation of fundamentally novel techniques for hardware based semantic-driven joint optimization of multimedia compression strategies and MU-MIMO transmissions in the context of resource-limited wireless systems. The PIs will leverage the support of this project to involve minority and underrepresented students in research and outreach activities. As part of the project, graduate students will develop unique expertise at the crossroads of machine learning, embedded systems and wireless networks.

Real-time object Detection with Edge Task offloading

The key technical efforts of this project will focus on the design of novel deep reinforcement learning (DRL)-based strategies that will control how the acquired data stream is compressed and wirelessly transmitted to the edge servers through MU-MIMO. The PIs will utilize techniques based on split computing to avoid increasing computational overhead due to the compression and MU-MIMO channel state information (CSI) feedback, while keeping the task accuracy close to the original. A full-fledged drone-based prototype based on customized software-defined radio (SDR) interfaces based on FPGA real-time processing and edge computing will be developed as part of the project. Large-scale data collection campaigns will be performed with a 64-antenna SDR testbed at Northeastern, a drone experimental testbed at UC Irvine, and the AERPAW PAWR platform to (i) collect the necessary wireless/multimedia data to train our algorithms; (ii) perform extensive testing and performance evaluation.

Personnel

Principal Investigator: Francesco Restuccia (Northeastern)
Co-Principal Investigator: Marco Levorato (UC Irvine)
Graduate Research Assistant: Foysal Haque (Northeastern)
Graduate Research Assistant: Sharon Ladrón de Guevara (UC Irvine)

Publications

F. Malandrino, G. Di Giacomo, A. Karamzade, M. Levorato, and C. Chiasserini, “Tuning DNN Model Compression to Resource and Data Availability in Cooperative Training'”, IEEE/ACM Transactions on Networking, 2023.

Y. Wu, M. Levorato, C. F. Chiasserini, “Tensor Compression and Reconstruction in Split DNN for Real-time Object Detection at the Edge”, 2024 IEEE International Mediterranean Conference on Communications and Networking (MeditCom).

T. Johnsen, Z. Xia, I. Harshbarger, and M. Levorato, “NaviSplit: Dynamic Multi-Branch Split DNNs for Efficient Distributed Autonomous Navigation”, IEEE 25th International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM), Perth, 2024.

T. Johnsen and M. Levorato, “NaviSlim: Adaptive Context-Aware Navigation and Sensing via Dynamic Slimmable Networks”, 9th ACM/IEEE Conference on Internet of Things Design and Implementation (IoTDI 2024).

M. Mendula, S. L.G. Contreras, M. Levorato and P. Bellavista, “Furcifer: a Context Adaptive Middleware for Real-world Object Detection exploiting Local, Edge, and Split Computing in the Cloud Continuum”, IEEE International Conference on Pervasive Computing and Communications (IEEE Percom), 2024, Biarritz, France, March 11-15, 2024.

D. Uvaydov, M. Zhang, C. P. Robinson, S. D’Oro, T. Melodia and F. Restuccia, “Stitching the Spectrum: Semantic Spectrum Segmentation with Wideband Signal Stitching,” Proc. of IEEE Conference on Computer Communications (IEEE INFOCOM), Vancouver, Canada, May 2024. DOI: 10.1109/INFOCOM52122.2024.10621332.

C. Puligheddu, N. Varshney, T. Hassan, J. Ashdown, F. Restuccia and C. F. Chiasserini, “OffloaDNN: Shaping DNNs for Scalable Offloading of Computer Vision Tasks at the Edge,” Proc. of the IEEE International Conference on Distributed Computing Systems (IEEE ICDCS), Jersey City, New Jersey, USA, July 23 – 26, 2024, pp. 624-634. DOI: 10.1109/ICDCS60910.2024.00064.

S. Rifat, M. De Lucia, A. Swami, J. Ashdown, K. Turck and F. Restuccia, “ADA: Adversarial Dynamic Test Time Adaptation in Radio Frequency Machine Learning Systems”, Proceedings of IEEE Military Communications Conference (IEEE MILCOM), 2024.

M. Zhang, M. De Lucia, A. Swami, J. Ashdown, K. Turck and F. Restuccia, “HyperAdv: Dynamic Defense Against Adversarial Radio Frequency Machine Learning Systems”, Proceedings of IEEE Military Communications Conference (IEEE MILCOM), 2024.

S. Rifat, J. Ashdown and F. Restuccia, ”DARDA: Domain-Aware Real-Time Dynamic Neural Network Adaptation,” Proceedings of IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Tucson, Arizona, 2025.

C. Puligheddu, J. Ashdown, C. F. Chiasserini, and F. Restuccia, “SEM-O-RAN: Semantic O-RAN Slicing for Mobile Edge Offloading of Computer Vision Tasks,” IEEE Transactions on Mobile Computing (IEEE TMC), Vol.: 23, Is: 7, pp. 7785 – 7800, July 2024. DOI: 10.1109/TMC.2023.3339056.

F. Busacca, S. Mangione, S. Palazzo, F. Restuccia, and I. Tinnirello, “SDR-LoRa, an Open-Source, Full-Fledged Implementation of LoRa on Software-Defined-Radios: Design and Potential Exploitation”, Computer Networks (COMNET), Volume 241, 110194, March 2024. DOI: 10.1016/J.COMNET.2024.110194.

F. Meneghello, F. Restuccia and M. Rossi, “WHACK: Adversarial Beamforming in MU-
MIMO Through Compressed Feedback Poisoning,” IEEE Transactions on Wireless Communications (IEEE TWC), 2024. DOI: 10.1109/TWC.2024.3452438.

Y. Wu, C. Chiasserini, F. Malandrino and M. Levorato, “Invited Paper: Enhancing Privacy in Federated Learning via Early Exit”, Proceedings of ACM Workshop on Advanced tools, programming languages, and PLatforms for Implementing and Evaluating algorithms for Distributed systems (ACM ApPLIED), 2023.

F. Malandrino, G. Di Giacomo, A. Karamzade, M. Levorato, and C. Chiasserini, “Matching DNN Compression and Cooperative Training with Resources and Data Availability”, Proceedings of IEEE Conference on Computer Communications (IEEE INFOCOM), 2023.

A. Alsoliman, F. Abkenar, and M. Levorato, “State-Recovery Protocol for URLLC Applications in 5G Systems”, IEEE Topical Conference on Wireless Sensors and Sensor Networks (IEEE WiSNet), 2023.

N. Bahadori, Y. Matsubara, M. Levorato and F. Restuccia, “SplitBeam: Effective and Efficient Beamforming in Wi-Fi Networks Through Split Computing”, Proceedings of IEEE International Conference on Distributed Computing Systems (IEEE ICDCS), 2023.

A. Coletta, F. Giorgi, G. Maselli, M. Prata, D. Silvestri, J. Ashdown and F. Restuccia, “AA-UAV: Application-Aware Content and Network Optimization of Edge-Assisted UAV Systems,” Proceedings of IEEE Conference on Computer Communications (IEEE INFOCOM), 2023.

M. Cominelli and F. Gringoli and F. Restuccia, “Exposing the CSI: A Systematic Investigation of CSI-based Wi-Fi Sensing Capabilities and Limitations,” Proc. of IEEE International Conference on Pervasive Computing and Communications (IEEE PerCom), 2023.

K. Foysal Haque, F. Meneghello, and F. Restuccia, “Wi-BFI: Extracting the IEEE 802.11 Beamforming Feedback Information from Commercial Wi-Fi Devices,” Proc. of ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization (ACM WINTECH), 2023.

Y. Matsubara, M. Levorato and F. Restuccia, “Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challenges,” ACM Computing Surveys (CSUR), Vol. 55, Is. 5, Art. No.: 90, pp. 130, December 2022.

F. Meneghello, C. Chen, C. Cordeiro and F. Restuccia, “Toward Integrated Sensing and Communications in IEEE 802.11bf Wi-Fi Networks,” IEEE Communications Magazine (IEEE COMMAG), Vol. 61, Is. 7, pp. 128 – 133, July 2023.

A. Pinto, A. Ashdown, T. Hassan, H. Cheng, F. Esposito, L. Bonati, S. D’Oro, T. Melodia, F. Restuccia, “Hercules: An Emulation-Based Framework for Transport Layer Measurements over 5G Wireless Networks,” Proceedings of ACM Workshop on Wireless Network Testbeds, Experimental evaluation & CHaracterization (ACM WINTECH), 2023.

C. Puligheddu, J. Ashdown, C. F. Chiasserini, and F. Restuccia, “SEM-O-RAN: Semantic and Flexible O-RAN Slicing for NextG Edge-Assisted Mobile Systems,” Proceedings of IEEE Conference on Computer Communications (IEEE INFOCOM), 2023.

F. Raviglione, C. Casetti and F. Restuccia, “Edge-V: Enabling Vehicular Edge Intelligence in Unlicensed Spectrum Bands“, Proceedings of IEEE Vehicular Technology Conference (VTC2023-Spring), 2023.

F. Restuccia, E. Blasch, A. Ashdown, J. Ashdown, and K. Turck, “3D-O-RAN: Dynamic Data Driven Open Radio Access Network Systems,” Proceedings of IEEE Military Communications Conference (IEEE MILCOM), pp. 19-24, 2022.

R. Rusca, F. Raviglione, C. Casetti, P. Giaccone, and F. Restuccia, “Mobile RF Scenario Design for Massive-Scale Wireless Channel Emulators,” Proceedings of Joint European Conference on Networks and Communications & 6G Summit (EuCNC/6G Summit), 2023.

Y. Matsubara, D. Callegaro, S. Singh, M. Levorato, and F. Restuccia, “BottleFit: Learning Compressed Representations in Deep Neural Networks for Effective and Efficient Split Computing,” Proceedings of IEEE International Symposium on a World of Wireless, Mobile, and Multimedia Networks (IEEE WoWMoM), 2022.

D. Callegaro, M. Levorato and F. Restuccia, “SmartDet: Context-Aware Dynamic Control of Edge Task Offloading for Mobile Object Detection,” Proceedings of IEEE International Symposium on a World of Wireless, Mobile, and Multimedia Networks (IEEE WoWMoM), 2022.

N. Bahadori, J. Ashdown and F. Restuccia, “ReWiS: Reliable Wi-Fi Sensing Through Few-Shot Multi-Antenna Multi-Receiver CSI Learning,” Proceedings of IEEE International Symposium on a World of Wireless, Mobile, and Multimedia Networks (IEEE WoWMoM), 2022. Best Paper Award.

F. Meneghello, M. Rossi and F. Restuccia, “DeepCSI: Rethinking Wi-Fi Radio Fingerprinting Through MU-MIMO CSI Feedback Deep Learning,” Proceedings of IEEE International Conference on Distributed Computing Systems (IEEE ICDCS), 2022.

P. Tehrani, F. Restuccia, and M. Levorato, “Federated Deep Reinforcement Learning for the Distributed Control of NextG Wireless Networks,” Proceedings of IEEE International Symposium on Dynamic Spectrum Access Networks (IEEE DySPAN), December 2021.

Patents

F. Restuccia, K. F. Haque and  M. Zhang, “Method and Apparatus for Wi-Fi Sensing Through MU-MIMO Beamforming Feedback Learning,” PCT. Application No.: PCT/US23/31651, filed on August 31, 2023.

F. Meneghello, M. Rossi and F. Restuccia, “System and Method for Identifying a Remote Device,” U.S. No.: 17/934,802, filed on September 23, 2022.

N. Bahadori and F. Restuccia, “System and Method for Sensing an Environment,” U.S. No.: 17/822,313, Filed on August 25, 2022.

F. Raviglione, F. Restuccia and C. Casetti, “Vehicular  Edge Intelligence in Unlicensed Spectrum  Bands”, PCT/US2023/064848, filed on March 23, 2023

N. Bahadori, Y. Matsubara, M. Levorato and F. Restuccia, “Beamforming in Wireless Networks Via Split Computing”, PCT Application No. PCT/US23/65300, filed on April 4, 2023.

C. Puligheddu, F. Restuccia, C.F. Chiasserini, “SEM-O-RAN: Semantic NextG O-RAN Slicing for Data-Driven Edge-Assisted Mobile Applications”, PCT Application No.: PCT/US2023/064901, filed on March 24, 2023. 

F. Restuccia, K. F. Haque and  M. Zhang, “SiMWiSense: Simultaneous Multi-Subject Activity Classification Through Wi-Fi Signals,” U.S. Application No.: 63/380,254,  filed on October 21, 2022.

S. Rifat, J. Ashdown, K. Turck and F. Restuccia, “DAZDA: Domain-Aware Zero-Shot Dynamic Adaptation of Neural Networks in Edge Computing,” U.S. Application No.: 63/580,492,  filed on Sept 5, 2023.

Educational Activities and Outreach

Master students involvement: a team of 6 master students is expanding the evaluation of our split computing algorithms to a wider set of autonomous platforms (e.g., rovers) to enable a comprehensive evaluation of data representation strategies in response to different contexts.

Undergrad and Junior students involvement: several undergraduate students (of which 5from minorities) and senior high-school student has been involved in the development of autonomous navigation algorithms.

NSF Abstract: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2134973&HistoricalAwards=false

CCRI: Principled Dataset Generation, Sharing and Maintenance Tools for the Wireless Community

Applied machine learning (ML) research in wireless faces challenges due the inability of domain experts to easily access existing well-curated, well-structured, and open-access datasets. Furthermore, there is a lack of direct access to a software framework that automates dataset creation and distribution based on detailed user requirements. RFDataFactory is a collaborative project that brings together investigators from Northeastern University and Rice University to bridge this gap. RFDataFactory aims to make available categorized datasets suitable for research related to ML in 5G and beyond networks, and advance fundamental understanding and design tools for accessing, creating, sharing and storing wireless datasets.

RFDataFactory will enable easy collection and preprocessing of physical layer to packet-level datasets through high-level directives and application programming interfaces. This will enable dataset generation for several NSF-funded experimentation platforms, such as the Colosseum emulator and NSF Platforms for Advanced Wireless Research. The project will significantly advance autonomous statistical analysis of RF spectrum activity, which will reduce data storage needs. Moreover, it will create pre-processing tools for removing device identifying information and facilitate generating standards compliant metadata headers. The project will also result in a search-able, centralized repository of both project-supported and user-contributed datasets with the focus on re-usability.

RFDataFactory will accelerate interdisciplinary research at the intersection of machine learning and the wireless domain, as well as bridging different communities and train a new generation of professionals for wireless dataset creation and sharing. The project will seek to involve underrepresented students in research and learning activities, support annual dataset gathering challenges, update advanced course materials with hands-on tutorials and laboratory sessions. Through targeted high-school outreach, the project will increase awareness and excitement in the next generation of researchers. The project will also generate value for other large-scale infrastructure investments already made by the NSF.

Project Url: All datasets, meta-data files, software application programming interfaces, tutorial materials, webinar recordings and other digital outcomes of this project will be maintained for 3 years, accessible via the project website after the completion of the project.

NSF Abstract: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2120447&HistoricalAwards=false

Securing Comparmented Information With Smart Radio Systems

The SCISRS program seeks to develop smart radio techniques to automatically detect and characterize RF anomalies that may indicate a compromise of secure data in complex RF environments. The specific types of anomalies include low probability of intercept (LPI) signals, altered or mimicked signals, and abnormal unintended emissions.

National security missions need to generate, store, use, transmit, and receive information and data both in secure facilities and “in the wild.” Data security is vital regardless of where the data are being used. Significant U.S. Government and private sector infrastructure investments have provided a high level of confidence in data security at facilities under the control of the data owner. However, data security is more challenging in environments where there is potentially much less control, such as those illustrated to the right. One possible indicator of attempted data breach is unexpected RF transmissions The goal of the SCISRS program is to develop smart radio techniques to automatically detect and characterize these suspicious signals and other RF anomalies in complex RF environments.The techniques must be scalable, computationally efficient, and adaptable to a range of radio hardware. IARPA’s partners have established RF testbeds to evaluate algorithms at the end of each program phase. These testbeds will be replete with ordinary overt signals to form two kinds of complex RF backgrounds the Intelligence Commuity may operate in. Into these backgrounds, the test partners will inject various RF anomalies. In phase I, the theme will be LPI signals which are designed to be hard to detect and characterize. In phase II, the theme will be otherwise ordinary signals that have been altered in some way to carry additional information or signals that have been designed to mimic ordinary overt signals. In phase III, the theme will be anomalous emanations which are typically emitted by electronic devices (e.g., monitors, keyboards) unintentionally, thereby transmitting potentially secure information by mistake.

Project website: https://www.iarpa.gov/research-programs/scisrs

RINGS: Internet of Things Resilience through Spectrum-Agile Circuits, and Maintenance Tools for the Wireless Community

As the Internet of Things (IoT) continues to grow at a fast pace, the increasing number of wireless devices in the frequency spectrum up to 6 GHz creates a compelling need to securely adapt IoT communications based on availability in the wireless spectrum environment. Autonomous coordination of wireless transmissions to avoid congestions becomes particularly important when numerous IoT devices with stringent power consumption restrictions communicate with an edge device connected to the cloud; collecting information with relatively low data rates such as biomedical signals, detected gases/chemicals levels, temperature, humidity, or vibration data. Such low-power IoT device applications include medical and health care, smart homes, transportation, manufacturing, agriculture, and environmental monitoring. It is imperative to design IoT networks with resilience features deeply embedded across layers from the integrated circuit level to the wireless system level. When IoT devices are employed with sensors in increasingly crowded environments to transmit sensed information, it is essential to increase their awareness of incumbent spectrum users and avoid interference. An overarching goal of this project is to create spectrum-agile IoT networks with low-power adaptive radio frequency (RF) circuits at the sensor nodes, and with coordinated optimization and enhanced security at the edge device. The synergies between the circuits, computing, and wireless networking components of this research are anticipated to create a paradigm for resilient next-generation IoT networks with energy-efficient secure communication between sensor nodes and edge devices. Research and education will be integrated by incorporating the obtained knowledge into graduate and undergraduate education. In addition, high school interns will be engaged through the Center for STEM Education at Northeastern University.

The project entails the research and development of a coordinated cross-layer design methodology for agile communication between edge devices and IoT sensor nodes. This is achieved by distributing spectrum sensing and real-time reconfiguration as follows: fast coarse spectrum sensing and reconfiguration in the sub-6 GHz frequency range on the analog/RF circuit level within low-power IoT devices, fine carrier sensing and network level optimizations on the edge device, and enhancement of high-level authentication and anomaly detection with the computing capabilities on the edge device; all aided by wirelessly transmitted information from temperature sensors used as activity detectors embedded in the IoT device transceiver. This cross-layer approach aims at enabling adaptive edge networks by providing the device-level ability to quickly respond to disruptive interference events by changing the transmit and receive frequencies at the IoT nodes, while performing intelligent real-time machine learning (ML) functions for coordinated communication within the network on the edge device with a software-defined radio (SDR) and field-programmable gate array (FPGA). Security will be enhanced at the wireless system level through ML-based RF fingerprinting, while robustness will be enhanced through federated learning techniques. At the hardware level, security will be enhanced through monitoring of power dissipation via embedded temperature sensors. The cross-cutting approach is not only expected to increase the component-level trust that can be established when new IoT devices are introduced into the network, but also to improve run-time reliability by capturing abnormal operations due to malicious intrusions or hardware faults based on the wirelessly transmitted on-chip temperature profiles from the IoT devices.

NSF Abstract: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2146754&HistoricalAwards=false

CC*: A Software-Defined Edge Infrastructure Testbed for Full-stack Data-Driven Wireless Network Applications

Interdisciplinary research advances often require devices to collect, process, and transfer large scientific datasets over high bandwidth links. The overarching goal of this project is to build a wireless virtual network testbed at Saint Louis University, in collaboration with Northeastern University, to evaluate network management solutions that integrate the use of machine learning and artificial intelligence with programmable radios and programmable network switches. To evaluate the proposed innovation in computer networking, the cyberinfrastructure will be used to prototype network protocols and systems in support of a few interdisciplinary initiatives on campus.

In particular, this project’s contributions will be developed around the integration of learning techniques with network mechanisms such as medium access control, routing, and transport services. First, the team will explore the design and implementation of effective transport and routing protocols that integrate the network stack at different scopes using recent advances in reinforcement learning. Second, novel network architectures will be proposed integrating edge network mechanisms with federated and split learning techniques. Third, cross-layer distributed learning protocols will be designed to create self-adaptive wireless networks. Such solutions will be tested on campus and on other network testbeds.

By combining synergies from the fields of data science and network virtualization protocols and architectures, this work will lay the foundation for further research in adaptive resource management for (wireless) edge computing applications that can improve the quality of life in our society. This project’s results will be valuable for other fields interested in real-time prediction, such as robotics, medicine, anthropology, and finance. The research in this project will be impactful also thanks to the planned industry and international collaborations. Students from underrepresented groups will be involved with research activities and hackathon events on campuses in Missouri and Maine.

The project will have a web presence at: https://cs.slu.edu/testbed/. Such website will be maintained by the Computer Science Department at Saint Louis University, and will be active at least 5 years beyond the end date of this project. The website will contain links to datasets collected with the testbed, technical reports, scientific publications, and code repositories developed by students and collaborators.

This award reflects NSF’s statutory mission and has been deemed worthy of support through evaluation using the Foundation’s intellectual merit and broader impacts review criteria.

NSF Abstract: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2201536&HistoricalAwards=false