A07: Scalable Parallel Scripting in the Cloud
SessionPoster Reception
Author
Event Type
ACM Student Research Competition
Poster
Reception
TimeTuesday, November 14th5:15pm -
7pm
LocationFour Seasons Ballroom
DescriptionIt’s often complicated, time consuming, but frequently
necessary to successfully port complex workflows to
multiple high-performance environments. Parsl is a
Python-based parallel scripting library that provides a
simple model for describing and executing dataflow-based
scripts over arbitrary execution resources such as
clouds, campus clusters, and high-performance systems.
Parsl’s execution layer abstracts the differences
between providers enabling provisioning and management
of compute nodes for use with a pilot system. In this
poster, we describe the development of a new execution
provider designed to support Amazon Web Services (AWS)
and Microsoft’s Azure. This provider supports the
transparent execution of implicitly parallel
Python-based scripts using elastic cloud resources. We
demonstrate that Parsl is capable of executing thousands
of applications per second over this elastic execution
fabric.
