Data processing networks made easy - improving development possibilities for people with limited computer science knowledge.
Master's thesis, University of Twente.
Full text available as:
Data processing networks are not only used by computer science people, nowadays researchers in all kinds of research topics are processing large amounts of data. Considering the required knowledge and amount of time, the development of a distributed and/or (de)centralized data processing network is often not an option for them. Sometimes they create scripts for the various processing steps, which are manually executed step-by-step. It is clear that such a procedure is far from optimal, especially in a streaming data environment. Due to the popularity
of the internet, distributed data or even decentralized processing is used more often. By introducing decentralization or distributed data, the complexity
of the overall processing network is increased pramatically. Both aspects require some overhead which lowers the understandability of the data processing
network in general.
We consider the following problems in supporting the development of data processing networks by people with limited computer science knowledge. First, a large group of users does not have a good overall understanding of the key concepts and reusable components of a data processing networks. Second, existing architectural styles are not well-suited for documenting a data processing view.
Third, the current tool support is insufficient for people with limited computer science knowledge.
To tackle the first problem, we introduce a basic model for data processing networks. Implementing a data processing network is a costly-process, it is important that the users have a good understanding of the general structure.
This basic model can be used to overcome a potential knowledge gap between the users and developers. To create a more detailed (technical) view of a data processing network, we introduce a specialized model.
As a new architectural style, we introduce the data processing style for documenting a data processing view. For documenting a more detailed (technical) view of a data processing network, we introduce the DatProNet style. By creating a specialized data processing view, the understandability of the general structure is increased.
As a solution for the third problem, we propose the DatProNet framework that supports the development of data processing networks by people with limited computer science knowledge. The complete development life cycle of data processing networks is covered by the DatProNet framework, which reduces the development and maintenance effort and increases the ease of use. XML-editors can be used for the creation, and validation, of architectural descriptions. Based on a valid architectural description, the framework can generate a concrete implementation.
The DatProNet framework provides a set of reusable components, which can be used to extend the framework if needed. Communication with external systems is possible through the use of serializers. The framework is completely implemented in JAVA, which implies a high portability.
|Item Type:||Master's Thesis|
|Research Group:||EWI-DB: Databases, EWI-SE: Software Engineering|
|Research Program:||CTIT-ASSIST: Applied Science of Services for Information Society Technologies|
|Research Project:||SensorDataLab: a sensor network infrastructure|
|Uncontrolled Keywords:||Stream processing, e-science, pipe and filter style|
|Deposited On:||10 October 2011|
Export this item as:
To correct this item please ask your editor
Repository Staff Only: edit this item