BEGIN:VCALENDAR VERSION:2.0 PRODID:IEEE vTools.Events//EN CALSCALE:GREGORIAN BEGIN:VTIMEZONE TZID:America/New_York BEGIN:DAYLIGHT DTSTART:20260308T030000 TZOFFSETFROM:-0500 TZOFFSETTO:-0400 RRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3 TZNAME:EDT END:DAYLIGHT BEGIN:STANDARD DTSTART:20261101T010000 TZOFFSETFROM:-0400 TZOFFSETTO:-0500 RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11 TZNAME:EST END:STANDARD END:VTIMEZONE BEGIN:VEVENT DTSTAMP:20260723T020110Z UID:5E6AC851-3DF0-4B21-AC2B-873BE8EC16AD DTSTART;TZID=America/New_York:20260718T143000 DTEND;TZID=America/New_York:20260718T163000 DESCRIPTION:Abstract: Generating accurate image captions and retrieving sem antically matched image-text pairs are typically treated as independent pr oblems\, limiting the depth of visual-semantic understanding a model can d evelop. This talk presents ViTCap-R\, a unified framework that integrates a Vision Transformer encoder\, a patch-level attention LSTM decoder\, and a dual-encoder contrastive retrieval module into a single cohesive pipelin e. By attending over 196 spatial patch embeddings during decoding\, the mo del produces interpretable captions grounded in specific image regions. A shared 256-dimensional embedding space\, trained with symmetric InfoNCE lo ss and hard negative mining\, enables effective cross-modal retrieval and caption reranking at inference time.\n\nWe evaluate ViTCap-R on Flickr8k a nd MS COCO 2014\, reporting BLEU\, METEOR\, and Recall@K metrics alongside qualitative attention heatmaps\, PCA projections\, and t-SNE visualizatio ns of the learned embedding space. Results demonstrate that unified traini ng improves semantic alignment between modalities\, while patch-level atte ntion offers meaningful interpretability gains over standard global-featur e baselines. We also discuss the challenges of scaling to high-diversity d atasets and the trade-offs introduced by hard negative mining\, pointing t oward future directions involving transformer-based decoders and larger-sc ale pretraining.\n\nSpeaker(s): Prayash Das\n\nAgenda: \nHybrid event\, in -person or online:\n2:30 Introduction\nTechnical Talks\nQuestion and Answe r\nNetworking\n4:30 Conclusion\n\nRoom: Meeting Rooms 2\,3\, 2 Civic Cent er Drive\, East Brunswick\, New Jersey\, United States\, 08816\, Virtual: https://events.vtools.ieee.org/m/558867 LOCATION:Room: Meeting Rooms 2\,3\, 2 Civic Center Drive\, East Brunswick\ , New Jersey\, United States\, 08816\, Virtual: https://events.vtools.ieee .org/m/558867 ORGANIZER:a.j.patel@ieee.org SEQUENCE:23 SUMMARY:ViTCap-R: Unifying Image Captioning and Cross-Modal Retrieval with Vision Transformers and Patch-Level Attention URL;VALUE=URI:https://events.vtools.ieee.org/m/558867 X-ALT-DESC:Description: <br /><p>Abstract: Generating accurate image captio ns and retrieving semantically matched image-text pairs are typically trea ted as independent problems\, limiting the depth of visual-semantic unders tanding a model can develop. This talk presents ViTCap-R\, a unified frame work that integrates a Vision Transformer encoder\, a patch-level attentio n LSTM decoder\, and a dual-encoder contrastive retrieval module into a si ngle cohesive pipeline. By attending over 196 spatial patch embeddings dur ing decoding\, the model produces interpretable captions grounded in speci fic image regions. A shared 256-dimensional embedding space\, trained with symmetric InfoNCE loss and hard negative mining\, enables effective cross -modal retrieval and caption reranking at inference time.</p>\n<p>We evalu ate ViTCap-R on Flickr8k and MS COCO 2014\, reporting BLEU\, METEOR\, and Recall@K metrics alongside qualitative attention heatmaps\, PCA projection s\, and t-SNE visualizations of the learned embedding space. Results demon strate that unified training improves semantic alignment between modalitie s\, while patch-level attention offers meaningful interpretability gains o ver standard global-feature baselines. We also discuss the challenges of s caling to high-diversity datasets and the trade-offs introduced by hard ne gative mining\, pointing toward future directions involving transformer-ba sed decoders and larger-scale pretraining.</p>\n<p>&nbsp\;</p><br /><br /> Agenda: <br /><p>Hybrid event\, in-person or online:<br>2:30 Introduction< br>Technical Talks<br>Question and Answer<br>Networking<br>4:30 Conclusion </p> END:VEVENT END:VCALENDAR