BEGIN:VCALENDAR
VERSION:2.0
PRODID:IEEE vTools.Events//EN
CALSCALE:GREGORIAN
BEGIN:VTIMEZONE
TZID:America/New_York
BEGIN:DAYLIGHT
DTSTART:20260308T030000
TZOFFSETFROM:-0500
TZOFFSETTO:-0400
RRULE:FREQ=YEARLY;BYDAY=2SU;BYMONTH=3
TZNAME:EDT
END:DAYLIGHT
BEGIN:STANDARD
DTSTART:20251102T010000
TZOFFSETFROM:-0400
TZOFFSETTO:-0500
RRULE:FREQ=YEARLY;BYDAY=1SU;BYMONTH=11
TZNAME:EST
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20260106T185711Z
UID:D0E1D5AD-3B88-467B-8B8C-D6A4FB9CECB5
DTSTART;TZID=America/New_York:20251205T130000
DTEND;TZID=America/New_York:20251205T140000
DESCRIPTION:The occurrence of bubbles in pipeline parallelism is an inheren
 t limitation that can account for more than 40% of the large language mode
 l (LLM) training time and is one of the main reasons for the underutilizat
 ion of GPU resources in LLM training. Harvesting these bubbles for GPU sid
 e tasks can increase resource utilization and reduce training costs but co
 mes with challenges. First\, because bubbles are discontinuous with variou
 s shapes\, programming side tasks becomes difficult while requiring excess
 ive engineering effort. Second\, a side task can compete with pipeline tra
 ining for GPU resources and incur significant overhead. To address these c
 hallenges\, we propose FreeRide\, a system designed to harvest bubbles in 
 pipeline parallelism for side tasks. FreeRide provides programmers with in
 terfaces to implement side tasks easily\, manages bubbles and side tasks d
 uring pipeline training\, and controls access to GPU resources by side tas
 ks to reduce overhead. We demonstrate that FreeRide achieves 7.8% average 
 cost savings with a negligible overhead of about 1% in training LLMs while
  serving model training\, graph analytics\, and image processing side task
 s.\n\nSpeaker(s): Sihang Liu\, \n\nRoom: 1304\, Bldg: DC\, 200 University 
 Ave W.\, Waterloo\, Ontario\, Canada\, N2L 3G1
LOCATION:Room: 1304\, Bldg: DC\, 200 University Ave W.\, Waterloo\, Ontario
 \, Canada\, N2L 3G1
ORGANIZER:mohammad.salahuddin@ieee.org
SEQUENCE:3
SUMMARY:FreeRide: Harvesting Bubbles in Pipeline Parallelism
URL;VALUE=URI:https://events.vtools.ieee.org/m/531575
X-ALT-DESC:Description: <br /><p>The occurrence of bubbles in pipeline para
 llelism is an inherent limitation that can account for more than 40% of th
 e large language model (LLM) training time and is one of the main reasons 
 for the underutilization of GPU resources in LLM training. Harvesting thes
 e bubbles for GPU side tasks can increase resource utilization and reduce 
 training costs but comes with challenges. First\, because bubbles are disc
 ontinuous with various shapes\, programming side tasks becomes difficult w
 hile requiring excessive engineering effort. Second\, a side task can comp
 ete with pipeline training for GPU resources and incur significant overhea
 d. To address these challenges\, we propose FreeRide\, a system designed t
 o harvest bubbles in pipeline parallelism for side tasks. FreeRide provide
 s programmers with interfaces to implement side tasks easily\, manages bub
 bles and side tasks during pipeline training\, and controls access to GPU 
 resources by side tasks to reduce overhead. We demonstrate that FreeRide a
 chieves 7.8% average cost savings with a negligible overhead of about 1% i
 n training LLMs while serving model training\, graph analytics\, and image
  processing side tasks.</p>
END:VEVENT
END:VCALENDAR