Parallel image generation on HPC systems via iLauncher

#Jupyter #Notebook #container #Simulations #RADAR #SAR #PHD #HPC-GPS

This work builds on another effort described in Application of Jupyter Notebook interfaces and iLauncher to deep learning workflows on HPC systems.Jupyter Notebook, container, Simulations, RADAR, SAR, PHD, HPC-GPS.  We describe a complex workflow application which generates millions of images in parallel on an HPC system via web interfaces using ipywidgets in Jupyter Notebooks and the Interface Launcher (iLauncher). Some computations are so complicated, taking many millions of HPC hours, that only a few subject matter experts are able to generate information efficiently. We present our custom application that walks the user through a workflow to include: target selection, configuration of the target, radar phase history simulation, and finally SAR image generation. The interface requests the user to enter a minimal set of parameters while other variables essential to computations are generated on the fly and provides status updates on workflow computations. Additionally, the ability to download any data component or view images interactively is provided. This application can be disconnected from the HPC system and reconnected at any time without slowing down the computations on the workflow submitted. Although typically a maximum run time must be specified when submitting a job to the queuing interface on an HPC system, this application uses the HPC-GPS tool to allow users to extend run times even after the initial request is submitted. Our new application helps to reduce the barrier to entry for both complicated physics-based simulations and using HPC systems.