parallel data processing

Dryad: Distributed data-parallel programs from sequential building blocks

Isard, M; Budiu, M; Yu, Y; A Birrell, D

Dryad is a general-purpose distributed execution engine for
coarse-grain data-parallel applications. A Dryad applica-
tion combines computational “vertices” with communica-
tion “channels” to form a dataflow graph. Dryad runs the
application by executing the vertices of this graph on a set of
available computers, communicating as appropriate through
files, TCP pipes, and shared-memory FIFOs.
The vertices provided by the application developer are
quite simple and are usually written as sequential programs
with no thread creation or locking. Concurrency arises from

Syndicate content