Efficient Hardware-Software Co-Design for AI with In-Memory Computing and HW-NAS Optimization

The rapid growth of AI and complex neural networks drives the need for efficient hardware that suits power and resource constraints. In-memory computing (IMC) is a promising solution for developing various IMC devices and architectures. Designing and deploying these systems requires a comprehensive hardware-software co-design toolchain that optimizes across devices, circuits, and algorithms. The Internet of Things (IoT) increases data generation, demanding advanced AI processing capabilities. Efficient deep learning accelerators, particularly for edge processing, benefit from IMC by reducing data movement costs and enhancing energy efficiency and latency, necessitating automated optimization of numerous design parameters.

Researchers from several institutions, including King Abdullah University of Science and Technology, Rain Neuromorphics, and IBM Research, have explored hardware-aware neural architecture search (HW-NAS) to design efficient neural networks for IMC hardware. HW-NAS optimizes neural network models by considering IMC hardware’s specific features and constraints, aiming for efficient deployment. This approach can also co-optimize hardware and software, tailoring both to achieve the most efficient implementation. Key considerations in HW-NAS include defining a search space, problem formulation, and balancing performance with computational demands. Despite its potential, challenges remain, such as a unified framework and benchmarks for different neural network models and IMC architectures.

HW-NAS extends traditional neural architecture search by integrating hardware parameters, thus automating the optimization of neural networks within hardware constraints like energy, latency, and memory size. Recent HW-NAS frameworks for IMC, developed since the early 2020s, support joint optimization of neural network and IMC hardware parameters, including crossbar size and ADC/DAC resolution. However, existing NAS surveys often overlook the unique aspects of IMC hardware. This review discusses HW-NAS methods specific to IMC, compares current frameworks, and outlines research challenges and a roadmap for future development. It emphasizes the need to incorporate IMC design optimizations into HW-NAS frameworks and provides recommendations for effective implementation in IMC hardware-software co-design.

In traditional von Neumann architectures, the high energy cost of transferring data between memory and computing units remains problematic despite processor parallelism. IMC addresses this by processing data within memory, reducing data movement costs, and enhancing latency and energy efficiency. IMC systems use various memory types like SRAM, RRAM, and PCM, organized in crossbar arrays to execute operations efficiently. Optimization of design parameters across devices, circuits, and architectures is crucial, often leveraging HW-NAS to co-optimize models and hardware for deep learning accelerators, balancing performance, computation demands, and scalability.

HW-NAS for IMC integrates four deep learning techniques: model compression, neural network model search, hyperparameter search, and hardware optimization. These methods explore design spaces to find optimal neural network and hardware configurations. Model compression uses techniques like quantization and pruning, while model search involves selecting layers, operations, and connections. Hyperparameter search optimizes parameters for a fixed network, and hardware optimization adjusts components like crossbar size and precision. The search space covers neural network operations and hardware design, aiming for efficient performance within given hardware constraints.

In conclusion, While HW-NAS techniques for IMC have advanced, several challenges remain. No unified framework integrates neural network design, hardware parameters, pruning, and quantization in a single flow. Benchmarking across various HW-NAS methods must be more consistent, complicating fair comparisons. Most frameworks focus on convolutional neural networks, neglecting other models like transformers or graph networks. Additionally, hardware evaluation often needs more adaptation to non-standard IMC architectures. Future research should aim for frameworks that optimize software and hardware levels, support diverse neural networks, and enhance data and mapping efficiency. Combining HW-NAS with other optimization techniques is crucial for effective IMC hardware design.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our Telegram Channel, Discord Channel, and LinkedIn Group.

If you like our work, you will love our newsletter..

Don’t Forget to join our 42k+ ML SubReddit