<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Big data processing often uses the paradigm of parallelism by computing directly on top of the distributed data storage. The existing big data workflows unify the data processing practices to utilize the cloud’s native computational potentials to offer advanced machine learning and BI capabilities. Spark is an open-source massively parallel in-memory data processing framework, the current state-of-the-art. The primary approach is to break down the job into granular-level executed tasks, enabling parallelization. In the discussed case study, through IoT – cloud solutions, the plant data can be converted into an analyzable form to let the farther machine learning modules produce added value. To maximize the efficiency of the processing and accumulation, cloud-based components are introduced. Based on the data insights, the appropriate operative actions can be taken. The cost and performance optimization methods were also discussed in the study. Through achieving higher degree of digitalization, the control over the production increased.</p>
</abstract>

Big data processing often uses the paradigm of parallelism by computing directly on top of the distributed data storage. The existing big data workflows unify the data processing practices to utilize the cloud’s native computational potentials to offer advanced machine learning and BI capabilities. Spark is an open-source massively parallel in-memory data processing framework, the current state-of-the-art. The primary approach is to break down the job into granular-level executed tasks, enabling parallelization. In the discussed case study, through IoT – cloud solutions, the plant data can be converted into an analyzable form to let the farther machine learning modules produce added value. To maximize the efficiency of the processing and accumulation, cloud-based components are introduced. Based on the data insights, the appropriate operative actions can be taken. The cost and performance optimization methods were also discussed in the study. Through achieving higher degree of digitalization, the control over the production increased.

Spark-Based Digital Factory Design

Acta Electrotechnica et Informatica

This work is licensed under the Creative Commons Attribution 4.0 International License.

{"article-title":"Spark-Based Digital Factory Design"}

Big data processing often uses the paradigm of parallelism by computing directly on top of the distributed data storage. The existing big data workflows unify...

Spark-Based Digital Factory Design

Online veröffentlicht: 12. Aug. 2022

Seitenbereich: 19 - 26

Eingereicht: 04. Apr. 2022

Akzeptiert: 20. Juni 2022

DOI: https://doi.org/10.2478/aei-2022-0008

Schlüsselwörter
Spark, big data, pipeline, cloud.ETL

© 2022 István Pölöskei, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Spark-Based Digital Factory Design

Online veröffentlicht: 12. Aug. 2022

Seitenbereich: 19 - 26

Eingereicht: 04. Apr. 2022

Akzeptiert: 20. Juni 2022

DOI: https://doi.org/10.2478/aei-2022-0008

SchlüsselwörterSpark, big data, pipeline, cloud.ETL

© 2022 István Pölöskei, published by Sciendo

This work is licensed under the Creative Commons Attribution 4.0 International License.

Schlüsselwörter
Spark, big data, pipeline, cloud.ETL