BEGIN:VCALENDAR VERSION:2.0 PRODID:IEEE vTools.Events//EN CALSCALE:GREGORIAN BEGIN:VTIMEZONE TZID:America/New_York BEGIN:DAYLIGHT DTSTART:20240310T030000 TZOFFSETFROM:-0500 TZOFFSETTO:-0400 RRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3 TZNAME:EDT END:DAYLIGHT BEGIN:STANDARD DTSTART:20231105T010000 TZOFFSETFROM:-0400 TZOFFSETTO:-0500 RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11 TZNAME:EST END:STANDARD END:VTIMEZONE BEGIN:VEVENT DTSTAMP:20231227T214049Z UID:A91A6189-5908-4437-95A0-3FF2030040EA DTSTART;TZID=America/New_York:20231219T212500 DTEND;TZID=America/New_York:20231219T223300 DESCRIPTION:IEEE Northern Jersey Section SMC Chapter Seminar\n\nUniADS: Uni versal Architecture-Distiller Search for Distillation Gap\n\n[]XIAOYU SEAN LU\, Ph.D. & Lecturer\n\nSchool of Cyber Science and Engineering\, Nanjin g University of Science and Technology\, Nanjing\, China\n\nTime: 9:30pm\, Dec. 19\n\nPlace: ECE 202\, NJIT\, Newark\, NJ\n\nhttps://njit.webex.com/ join/zhou\n\nAbstract\n\nIn this talk\, we present our proposed method cal led UniADS. It is the first Universal Architecture-Distiller Search framew ork for co-optimizing student architecture and distillation policies. Teac her-student distillation gap limits the distillation gains. Previous appro aches seek to discover the ideal student architecture while ignoring disti llation settings. In UniADS\, we construct a comprehensive search space en compassing an architectural search for student models\, knowledge transfor mations in distillation strategies\, distance functions\, loss weights\, a nd other vital settings. To efficiently explore the search space\, we util ize the NSGA-II genetic algorithm for better crossover and mutation config urations and employ the Successive Halving algorithm for search space prun ing\, resulting in improved search efficiency and promising results. Exten sive experiments are performed on different teacher-student pairs using CI FAR-100 and ImageNet datasets. The experimental results consistently demon strate the superiority of our method over existing approaches. Furthermore \, we provide a detailed analysis of the search results\, examining the im pact of each variable and extracting valuable insights and practical guida nce for distillation design and implementation.\n\nBio-Sketch\n\nXIAOYU SE AN LU received the B.S. degree from the Nanjing University of Technology\, Nanjing\, China\, in 2011\, and the M.S. and Ph.D. degrees from the New J ersey Institute of Technology\, Newark\, NJ\, USA\, in 2015 and 2019\, res pectively. He was a Research Scholar with the Department of Electrical and Computer Engineering\, Stevens Institute of Technology\, Hoboken\, NJ\, U SA. He joined the School of Cyber Science and Engineering\, Nanjing Univer sity of Science and Technology\, Nanjing\, China in 2022. He has published more than 20 articles in journals and conference proceedings\, including the IEEE TRANSACTIONS ON SYSTEM\, MAN AND CYBERNETICS: SYSTEMS\, the IEEE/ CAA JOURNAL OF AUTOMATICA SINICA\, and the IEEE TRANSACTIONS ON COMPUTATIO NAL SOCIAL SYSTEMS. His current research interests include social media da ta analysis\, cyberbullying detection\, knowledge distillation\, data mini ng\, deep learning\, and their applications in industry.\n\n323 MLK Blvd.\ , Newark\, New Jersey\, United States\, 07102\, Virtual: https://events.vt ools.ieee.org/m/394722 LOCATION:323 MLK Blvd.\, Newark\, New Jersey\, United States\, 07102\, Virt ual: https://events.vtools.ieee.org/m/394722 ORGANIZER:zhou@njit.edu SEQUENCE:12 SUMMARY:Seminar on UniADS: Universal Architecture-Distiller Search for Dist illation Gap URL;VALUE=URI:https://events.vtools.ieee.org/m/394722 X-ALT-DESC:Description: <br /><p>IEEE Northern Jersey Section SMC Chapter S eminar</p>\n<p><strong><em><u>&nbsp\;</u></em></strong></p>\n<table>\n<tbo dy>\n<tr>\n<td width="624">\n<p>UniADS: Universal Architecture-Distiller S earch for Distillation Gap</p>\n</td>\n</tr>\n<tr>\n<td width="624">\n<p>< a name="_Toc386797556"></a>XIAOYU SEAN LU\, Ph.D. &amp\; Lecturer</p>\n<p> School of Cyber Science and Engineering\, Nanjing University of Science an d Technology\, Nanjing\, China</p>\n<p>&nbsp\;</p>\n<p>Time: 9:30pm\, Dec. 19&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;& nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp \;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&nbsp\;&n bsp\;&nbsp\;&nbsp\;</p>\n<p>Place: ECE 202\, NJIT\, Newark\, NJ</p>\n<p><a href="https://njit.webex.com/join/zhou">https://njit.webex.com/join/zhou< /a></p>\n<p>&nbsp\;</p>\n<p><em>Abstract</em></p>\n<p>In this talk\, we pr esent our proposed method called UniADS. It is the first Universal Archite cture-Distiller Search framework for co-optimizing student architecture an d distillation policies. Teacher-student distillation gap limits the disti llation gains. Previous approaches seek to discover the ideal student arch itecture while ignoring distillation settings. In UniADS\, we construct a comprehensive search space encompassing an architectural search for studen t models\, knowledge transformations in distillation strategies\, distance functions\, loss weights\, and other vital settings. To efficiently explo re the search space\, we utilize the NSGA-II genetic algorithm for better crossover and mutation configurations and employ the Successive Halving al gorithm for search space pruning\, resulting in improved search efficiency and promising results. Extensive experiments are performed on different t eacher-student pairs using CIFAR-100 and ImageNet datasets. The experiment al results consistently demonstrate the superiority of our method over exi sting approaches. Furthermore\, we provide a detailed analysis of the sear ch results\, examining the impact of each variable and extracting valuable insights and practical guidance for distillation design and implementatio n.</p>\n<p>&nbsp\;</p>\n<p>Bio-Sketch</p>\n<p>XIAOYU SEAN LU received the B.S. degree from the Nanjing University of Technology\, Nanjing\, China\, in 2011\, and the M.S. and Ph.D. degrees from the New Jersey Institute of Technology\, Newark\, NJ\, USA\, in 2015 and 2019\, respectively. He was a Research Scholar with the Department of Electrical and Computer Engineeri ng\, Stevens Institute of Technology\, Hoboken\, NJ\, USA. He joined the S chool of Cyber Science and Engineering\, Nanjing University of Science and Technology\, Nanjing\, China in 2022. He has published more than 20 artic les in journals and conference proceedings\, including the IEEE TRANSACTIO NS ON SYSTEM\, MAN AND CYBERNETICS: SYSTEMS\, the IEEE/CAA JOURNAL OF AUTO MATICA SINICA\, and the IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS. His current research interests include social media data analysis\, cyber bullying detection\, knowledge distillation\, data mining\, deep learning\ , and their applications in industry.&nbsp\;</p>\n</td>\n</tr>\n</tbody>\n </table> END:VEVENT END:VCALENDAR