Enhancing Deep Learning Models through Tensorization: A Comprehensive Survey and Framework

Abstract

The rapid expansion of publicly available data and the growing complexity of deep learning models have highlighted the need for more effective data representation and analysis methods. Tensorization provides a revolutionary solution, aligning the multidimensional nature of data with compressed deep learning models to yield more interpretable results. This paper provides an in-depth, tutorial-style review of tensorization, multi-way analysis methods, and their integration with deep neural network models, illustrated through various case studies. A Blind Source Separation experiment compares the performance of 2-dimensional algorithms with multi-way algorithms. Experiments were conducted on multiple datasets under different noise and compression conditions. Results indicate that while traditional 2D methods achieve lower Root Mean Square Error, tensor-based methods preserve essential structural and frequency characteristics, making them valuable for applications when accurate signal reconstruction is required. Contrary to the expected difficulties of high dimensionality, utilising multidimensional datasets in their original form and applying multi-way analysis methods based on multilinear algebra can uncover complex relationships among dimensions while reducing model parameters and accelerating processing.