Development of a Scalable Silicon Photonic On-Chip Memory Architecture
By Sathvik Redrouthu, Pranav Vadde, Pranav Velleleth
Over the past decade, there has been a dramatic increase in the parameter count of neural networks, driven by advances in machine learning algorithms, hardware, and data availability. This increase has enabled significant improvements in performance on a wide range of tasks, from image classification to natural language processing. In 2012, the state-of-the-art image classification model, AlexNet, had only 61 million parameters. By 2015, the VGG-19 model had 143 million parameters, and by 2016, the ResNet-152 model had 60 million parameters. In 2018, the DenseNet-264 model had 36.4 million parameters, while the EfficientNet-B7 model, released in 2019, had 66 million parameters. The parameter counts of natural language models have also increased significantly. In 2015, the state-of-the-art language model had only 5 million parameters. By 2018, the OpenAI GPT-2 model had 1.5 billion parameters, and by 2020, the GPT-3 model had 175 billion parameters. This increase in parameter count has raised concerns about the computational cost and environmental impact of training and running these models. For example, training the GPT-3 model on a single accelerator can consume up to 1.2 GWh of electricity. To address these concerns, in addition to exploring various techniques for reducing the neural network parameter counts while maintaining performance, we have developed a set of silicon photonic accelerators with significantly higher speed and energy ratings for inference processes. Despite these advantages, these accelerators don’t have the capabilities to efficiently execute larger models, as data must be converted to electrical signals for traditional intermediate memory storage. Thus, we explore nonvolatile optical memory, with the goal of removing these intermediate conversions and improving overall performance. We begin by evaluating the effectiveness of on-chip silicon photonic memory architectures; notability, those using Phase Change Materials. We then turn to the original free-space experiments within nonvolatile optical memory and go on to design an experiment taking these to the silicon photonic domain. We address introduced issues like on-chip crosstalk through novel innovations to introduce scalability with optical memory cells, which was previously not possible. We finish by evaluating our on-chip memory system against conventional systems and other silicon photonic architectures in literature in “storage time”, “scalability”, “storage capacity”, and “read/write time.”