Monthly webinar: "Embracing the Resiliency of Deep Neural Networks: Rethinking Old Mechanisms" and "On a Recoverability of Graph Neural Network Representations"

#Deep #Neural #Networks #Graph #Network

RE: Monthly webinar - 2 Talks:

Talk 1: Embracing the Resiliency of Deep Neural Networks: Rethinking Old Mechanisms

Speaker 1: Dr. Gil Shomron, Electrical and Computer Engineering Faculty, Technion – Israel Institute of Technology.

Talk 2: On a Recoverability of Graph Neural Network Representations

Speaker 1: Dr. Chaim Baskin, Electrical and Computer Engineering Faculty, Technion – Israel Institute of Technology.

Date: Tuesday, March 22^nd, 2022 3:30 PM (Israel Time),

Dear IEEE Israel chapter members, Dear guests,

The IEEE Computer Society Israel conducts a series of webinars in different areas of computer systems, Software engineering, Computer architectures, data center, accelerators for machine learning, security, and more. The webinars offer insightful and enriching talks held by international leaders and professionals of the computer society sector.

The next free online webinar will include two talks:

1. “Embracing the Resiliency of Deep Neural Networks: Rethinking Old Mechanisms” by Dr. Gil Shomron, Electrical and Computer Engineering Faculty, Technion – Israel Institute of Technology.

Deep neural networks (DNNs) have gained tremendous momentum in recent years, both in academia and industry. Yet, DNNs are compute intensive and may require billions of multiply-and-accumulate operations for a single input query. Limited resources, such as those in IoT devices, latency constraints, and high input throughput, all drive research and development of efficient computing methods for DNN execution. In our research, we rethink two well-known CPU methods – simultaneous multithreading (SMT) and value prediction – and map them to the new environment introduced by DNNs, by leveraging their unique characteristics. With SMT, we propose a new concept of non-blocking SMT (NB-SMT), in which execution units are shared among several computational flows to avoid idle MAC operations due to zero-valued operands. We present and discuss the path from a data-driven “blocking” SMT design to the concept of NB-SMT, to a fine-tuned sparsity-aware quantization method. As for value prediction, we present prediction schemes which leverage the inherent spatial correlation in CNN feature maps to predict zero-valued activations. By speculating which activations will be zero-valued, we potentially reduce the required MAC operations.

2. “On a Recoverability of Graph Neural Network Representations” by Dr. Chaim Baskin, Electrical and Computer Engineering Faculty, Technion – Israel Institute of Technology.

Despite their growing popularity, graph neural networks (GNNs) still have multiple unsolved problems, including finding more expressive aggregation methods, propagation of information to distant nodes, and training on large-scale graphs. Understanding and solving such problems require developing analytic tools and techniques. In this work, we propose the notion of recoverability, which is tightly related to information aggregation in GNNs, and based on this concept, develop the method for GNN embedding analysis. We define recoverability theoretically and propose a method for its efficient empirical estimation. We demonstrate, through extensive experimental results on various datasets and different GNN architectures, that estimated recoverability correlates with aggregation method expressivity and graph sparsification quality. Therefore, we believe that the proposed method could provide an essential tool for understanding the roots of the aforementioned problems, and potentially lead to a GNN design that overcomes them.

The Webinar is free, but pre-registration is required. So, please sign up using the below link https://technion.zoom.us/webinar/register/WN_-vxCyWMpRO2sSVyJT2h40A and the Zoom session details will be provided after registration.

Please contact us for any further details and updates on the series of IEEE Computer Society Webinars.

We are looking forward to your participation and future collaboration.

Prof. Avi Mendelson                                                       Prof. Freddy Gabbay
Avi.mendelson@technion.ac.il                                     freddyg@ruppin.ac.il

Chairman                                                                        Vice-Chair