Application Massive Data Processing Platform for Smart Manufacturing Based on Optimization of Data Storage

2023 | ACM Transactions on Management Information Systems | Citations: 0

Authors: Ren, Bin; Chen, Yuquiang; Wang, Fujie

Abstract: The aim of smart manufacturing is to reduce manpower requirements of the product ... Expand

Abstract: The aim of smart manufacturing is to reduce manpower requirements of the production line by applying technology of huge amounts of data to the manufacturing industry. Smart manufacturing is also called Industry 4.0, and the platform for processing huge amounts of data has an indispensable role. The massive data processing platform is like the brain of the entire factory, receiving all data from production line sensors via edge computing, processing, and analyzing, and finally making feedback decisions. With the innovation of production technology, the data that the platform needs to process has become diverse and complex, and the amount has become increasingly large. As well, many precision manufacturing industries have begun to enter the field of Industry 4.0. In addition to the accuracy and availability of data processing, there is emphasis on the real-time nature of data processing. After the sensor receives the data, the platform must provide feedback within a short period of time. This article proposes a massive data processing platform based on the Lambda architecture, which has the coexistence of stream processing and batch processing to meet real-time feedback needs of high-precision manufacturing. To verify the effectiveness of the optimization, it is based on real data from the manufacturing industry. To generate a large amount of test data to confirm the optimization of the storage of pictures. The results show that it optimizes the storage and optimization of the image data generated by the Automated Optical Inspection technology used in manufacturing today and optimizes the query for data storage. It also reduces the consumption of a large amount of memory as expected, and the query for Hive reduced the time spent. Collapse

Topics: Apache HBase Apache Hadoop Hadoop Distributed File System stream processing Python

Methods: experiment

Combining Graph Exploration and Fragmentation for Scalable RDF Query Processing

2021 | Information Systems Frontiers | Citations: 0

Authors: Khelil, Abdallah; Mesmoudi, Amin; Galicia, Jorge; Bellatreche, Ladjel; Hacid, Mohand-Saïd; Coquery, Emmanuel

Abstract: The flexibility offered by the Resource Description Framework (RDF) has led it t ... Expand

Abstract: The flexibility offered by the Resource Description Framework (RDF) has led it to become a very popular standard for representing data with an undefined or variable schema using the concept of triples. Its success has resulted in many large scale multidisciplinary datasets, that have prompted the development of efficient RDF processing systems. Current approaches can be distinguished into two groups: the first, adopting the relational model storing the triples in tables, and the second creating data structures that model RDF data as a graph. The strategies of the first group are more easily scalable since they apply optimization strategies from the relational model like indexing and fragmentation. However, these approaches suffer many overheads when dealing with complex queries (e.g. compounded SPARQL graphs involving filters) persistent in existing applications. On the other hand, graph-based systems that use more complex data structures fail to efficiently manage the main memory and are not scalable in computer hardware with limited resources. In this paper, we propose a novel approach to perform queries (Basic Graph Patterns, Wildcards, Aggregations and Sorting) on RDF data. We propose to combine both RDF graph exploration with physical fragmentation of triples. In this work, we describe our graph-based storage and query evaluation models. Then, we detail the architecture of our system and we largely explain the strategy, based in the Volcano execution model, used to manage the main memory at query runtime. We conducted extensive experiments on synthetic and real datasets to evaluate the efficiency of our proposal. We compared our performance with a relational-based (Virtuoso), a graph-based (gStore) and an intensive-indexing (RDF-3X) approach. According to our evaluation, our system offers the best compromise between efficient query processing and scalability. Collapse

Topics: computer hardware Neo4j graph database structured query language database system

Methods: experiment computational algorithm leading question literature study

Applying Multi-Criteria Decision-Making for the Selection of Big Data Technologies

2021 | Americas Conference on Information Systems | Citations: 0

Authors: Volk, Matthias; Staegemann, Daniel; Bischoff, Dennis; Turowski, Klaus

Abstract: For many years now big data presents itself as a technical starting point for a ... Expand

Abstract: For many years now big data presents itself as a technical starting point for a multitude of different trends, including data science, artificial intelligence, the internet of things, industry 4.0, and others. However, many decision makers are still challenged by the realization of related projects. One of the major problems arises from the multitude of existing technologies as well their selection and application. Due to that, in this paper, applying the design science research methodology, a comprehensive investigation about the application of multi-criteria decision-making (MCDM) methods is conducted. Apart from a structured literature review that seeks to identify relevant requirements for the technology selection, a conceptual model, as well as a prototypical implementation in the form of a decision support system, are presented. In the future, further MCDM approaches shall be tested and the system implemented in larger processes, including general sensemaking, system architecture modeling and rapid deployment. Collapse

Topics: big data decision support decision support system Apache Hadoop electronic authentication

Methods: design artifact structured literature research experiment design science multiple criteria decision making

Efficient querying of multidimensional RDF data with aggregates: Comparing NoSQL, RDF and relational data stores

2020 | International Journal of Information Management | Citations: 0

Authors: Ravat, Franck; Song, Jiefu; Teste, Olivier; Trojahn, Cassia

Abstract: This paper proposes an approach to tackle the problem of querying large volume o ... Expand

Abstract: This paper proposes an approach to tackle the problem of querying large volume of statistical RDF data. Our approach relies on pre-aggregation strategies to better manage the analysis of this kind of data. Specifically, we define a conceptual model to represent original RDF data with aggregates in a multidimensional structure. A set of translations rules for converting a well-known multidimensional RDF modelling vocabulary into the proposed conceptual model is then proposed. We implement the conceptual model in six different data stores: two RDF triple stores (Jena TDB and Virtuoso), one graph-oriented NoSQL database (Neo4j), one column-oriented data store (Cassandra), and two relational databases (MySQL and PostGreSQL). We compare the querying performance, with and without aggregates, in these data stores. Experimental results, on real-world datasets containing 81.92 million triplets, show that pre-aggregation allows for reducing query runtime in all data stores. Neo4j NoSQL and relational databases with aggregates outperform triple stores speeding up to 99% query runtime. Collapse

Topics: Neo4j Cassandra relational database nosql database MySQL

Methods: experiment conceptual modelling computational algorithm multidimensional modeling optimization model

Coordination in OSS 2.0: ANT Approach

2019 | HICSS | Citations: 0

Authors: You, Sangseok; Crowston, Kevin; Saltz, Jeffrey S; Hegde, Yatish

Abstract: Open source software projects are increasingly driven by a combination of indepe ... Expand

Abstract: Open source software projects are increasingly driven by a combination of independent and professional developers, the former volunteers and the later hired by a company to contribute to the project to support commercial product development. This mix of developers has been referred to as OSS 2.0. However, we do not fully understand the multi-layered coordination spanning individuals, teams, and organizations. Using Actor-Network Theory (ANT), we describe how coordination and power dynamics unfold among developers and how different tools and artifacts both display activities and mediate coordination efforts. Internal communication within an organization was reported to cause broken links in the community, duplication of work, and political tensions. ANT shows how tools and code can exercise agency and alter a software development process as an equivalently active actor of the scene. We discuss the theoretical and practical implications of the changing nature of open source software development. Collapse

Topics: Apache Hadoop VMware anonymity pseudonymity user experience

Methods: qualitative interview personal interview grounded theory qualitative coding TensorFlow

Theories: actor network theory theory of affordance socio technical theory

Adaptive Big Data Analytics for Deceptive Review Detection in Online Social Media

2014 | International Conference on Information Systems | Citations: 4

Authors: Zhang, Wenping; Lau, Raymond; Li, Chunping

Abstract: The explosive growth of user-contributed reviews in e-Commerce and online social ... Expand

Abstract: The explosive growth of user-contributed reviews in e-Commerce and online social network sites prompts for the design of novel big data analytics frameworks to cope with such a challenge. The main contributions of our research are twofold. First, we design a novel big data analytics framework that leverages distributed computing and streaming to efficiently process big social media data streams. Second, we apply the proposed framework that is underpinned by a novel parallel co-evolution genetic algorithm to adaptively detect deceptive reviews with respect to different social media contexts. Our experiments show that the proposed big data analytics framework can effectively and efficiently detect deceptive reviews from a big social media data stream, and it outperforms other non-distributed big data analytics solutions. To the best of our knowledge, this is the first successful design of an adaptive big data analytics framework for deceptive review detection under a big data environment. Collapse

Topics: big data Twitter social media email spam electronic commerce

Methods: experiment support vector machine computational algorithm machine learning ensemble learning

Theories: co-evolution theory