Abstract
Caravela is a computing platform that has been designed in academia to implement distributed stream-based computing based on the flow-model. This paper is focused on the new meta-pipeline execution mechanism, which allows to set up a pipeline of flow-models to be executed in parallel but with data streams flowing in sequence along them. A model and programming tools are presented to develop the proposed meta-pipeline in the Caravela platform. These tools are important, namely to tackle the deadlock problem that can occur due to uninitialised input data streams. A new efficient algorithm is also proposed to identify off-line the uninitialised edges in a ‘pipeline graph’. Moreover, optimisation techniques are also proposed to reduce the computation time and the amount of memory required to implement the proposed algorithm, and a graphical user interface (GUI) was implemented to facilitate the programming of applications with the meta-pipeline approach. The experimental results presented in this paper show the practical interest of the meta-pipeline approach for stream-based computing and the effectiveness of both the proposed algorithm and the developed GUI for programming the Caravela platform.
Acknowledgement
This work is partially supported by the Portuguese Foundation for Science and Technology (FCT) through the FEDER program.