Evaluation of Cluster Management Quality Based on Consumer Opinion Sentiment Analysis

In the age of digitization and a rapidly changing environment, companies and organizations depend on up-to-date information about how consumers perceive their products and services. Continuous monitoring of consumer opinions and satisfaction levels leads, for example, to a better adjustment of product offerings to customer requirements and prevents image crises. Consumers also want to know what other users think about services or products before making a buying decision. This stems from a common practice of making decisions after reading opinions and confronting them with their expectations (Qazi, et al., 2017).

Until a few years ago, the most trusted source of opinions in the first place was family or friends (Perkins and Fenech, 2016). For businesses and organizations, the opposite was true. When they needed opinions regarding the products or services they offered, they conducted surveys, group interviews, and/or market research. It was a costly as well as time-consuming process and often the insight into why a product did not sell and consumers’ dissatisfaction was received too late. Businesses and organizations increasingly rely on social media, online reviews, online forums, blogs, microblogs, Twitter, and online comments and articles for decision-making (Liu, 2011), including those made at higher levels of the organization (Jeyapriya, et al., 2015).

With the increase in the number of Internet users, increasing importance has been given to the information published on social networks. Consumers are willing to share their opinions and thoughts about products and services (Pozzi, et al., 2016), and the lack of an online presence of a business does not preclude sharing opinions about it online. This has made it possible to analyze the quality of services and products provided by a company based on online reviews. The increasing popularity of this phenomenon among researchers is also observed. Several studies have analyzed the sentiment of consumer opinions in relation to the evaluation of quality of service (QoS) (Albesta, et al., 2021), for example, in the hotel (Duan, et al., 2013), restaurant (Nakayama, et al., 2019), and health-care industries (Abirami, et al., 2017). However, there is no research that could unequivocally establish a link between service quality evaluation and business management quality.

This issue concerns not only individual entrepreneurs, but also entire clusters of them, from suppliers to companies operating in related sectors, service units, or institutions related to them (Porter, 2001). Cluster management presents a challenge that many do not know how to meet; therefore, it is essential to assess cluster management quality. Initiatives to motivate better cluster management are being launched. Despite the existence of systems such as the European Cluster Assessment System, National Key Clusters, and Cluster Management Standards, there still remains a need for additional solutions that can rapidly and continuously assess cluster management quality.

The aforementioned consumer sentiment analysis may prove to be helpful in this respect.

The reasons discussed above prompted the authors of this paper to address this issue and pose a hypothesis that cluster management quality affects the level of customer business opinion sentiment regarding cluster members.

The hypothesis was tested by comparing the evaluation of cluster management quality, determined on the basis of sentiment analysis of online opinions of cluster and in-cluster business customers, with the level of cluster management quality measured by the European Cluster Excellence Initiative (ECEI) label. The topic is important because the ECEI evaluation system does not directly relate to the in-cluster businesses and the level of services they provide, and it is these businesses that can testify to the success of a cluster as well as the quality of its management.

The potential relationship between the evaluation of service quality and the quality of company/cluster management is important for cluster managers. It gives the possibility of current verification of management quality of cluster members, without waiting for periodic evaluation of ECEI award, because the quality of cluster management depends on the quality of management of cluster members. It can be a warning signal before the entry of the cluster into the transition phase, as well as a tool for verifying the quality of candidates for cluster members.

Reviews published online are a special type of opinion. The paper Bernatowicz and Małyszko (2014) pointed out that supporting friends in decision-making by creating a network of mutual recommendations and advice is a natural phenomenon not just among friends, but also among strangers, which partially transfers the bonding mechanisms to the Internet plane. Further reasons for consumer involvement in publishing opinions and reviews were analyzed in Bruhn, et al. (2004). It was pointed out that their purpose, besides helping, may be to harm a particular business or organization through negative reviews of its products or services that are not always truthful. It is difficult for many businesses to resist the temptation to publish manipulated reviews. For this purpose, they may resort to organized groups, which, in return for definite financial benefits, anonymously or under a changed identity, will publish unreliable information about competitors or unfounded praise of certain products or services (Liu, 2015). Some of the most common reasons for publishing reviews include the following (Directorate For Science, Technology And Innovation Committee On Consumer Policy, 2019):

the desire to help consumers in order to avoid a bad product or recommend a good one,

There are many documented cases of success of such actions. For example, the manufacturer of Doritos crisps discontinued the use of palm oil in products sold in Poland after a series of negative comments were posted on the manufacturer's website (Mandel, 2018). Similarly, the beer producer Heineken was forced to remove its commercial from its official YouTube channel after Internet users accused it of racism (Taylor, 2018). Therefore, it is important to identify emotions on the basis of comments posted by Internet users.

Sentiment analysis can be of help in this regard. It is a field of research that analyzes people's opinions, feelings, evaluations, attitudes, and emotions toward entities such as products, services, organizations, people, problems, events, topics, and their attributes (Liu, 2015). It is often interchangeably referred to as overtone analysis, sentiment analysis, opinion exploration, opinion mining, sentiment exploration, subjectivity analysis, emotion analysis, etc. Although the concept of sentiment analysis first appeared in a publication recently (Nasukawa, et al., 2003), several papers on this topic had already been published before (Das, et al., 2001; Pang, et al., 2002). Because of its importance to business and society as a whole, it has influenced the development of many sciences, from computer science to management and finance (Liu, 2015).

Sentiment analysis and opinion research papers revolve around assigning the research subject to one of the classes representing positive or negative overtones (Ravi and Ravi, 2015), so that it is possible to indicate the emotional states of an opinion writer as well as to determine the emotional effect that an opinion may have (Gładysz, 2017). This can be done in different manners; the Internet users most commonly use (Małyszko, 2015):

evaluation in the form of points or stars in the adopted scale and

In conjunction with the development of sentiment analysis and automatic Internet monitoring systems, an increasing number of studies have begun to be published that examine not only consumer product perceptions, but also health care, financial services, and social events such as presidential elections and bills enacted. For example, Liu, et al. (2007) proposed a sentiment analysis model for prediction of sales performance. One paper (McGlohon, et al., 2010) evaluated merchants through their reviews. A study on sentiment analysis of Twitter messages (Gimpe, et al., 2010) was juxtaposed with public opinion polls, while Tumasjan, et al. (2010) used Twitter to predict election results. Sentiment analysis was used by Dzieciątko (2018) to analyze emotions in political speeches in the Sejm, and Chen, et al. (2010) investigated political views through it. In their study, Asur, et al. (2010) and Sadikov Parameswaran, and Venetis (2009) analyzed Twitter opinions on movie reviews and then predicted ticket sales revenue based on them.

Research on the use of service quality assessment and sentiment analysis was conducted in the article Chong, et al. (2016), where the sales of products were predicted based on consumer opinions. In the paper by Archak, et al. (2011), the authors analyzed consumer preferences based on online reviews. In the study by Kauffmann, et al. (2020), sentiment analysis was used to assess the quality of higher education institutions, while in the research by Młodzianowski (2018), it was used to forecast the direction of change in stock market indexes.

In this study, the analysis of online opinions of cluster and in-cluster business customers will be compared with cluster management quality as measured by the level of the ECEI label. The ECEI was established by the European Commission – the Directorate General for Enterprise and Industry. The ECEI certification system was set up to provide independent, voluntary evidence of excellence in cluster management. Its purpose is not only to discern between “good” and “bad” cluster management systems, but also to motivate managers to join and sustainably participate in the improvement process. As a first step, the process involves benchmarking against other clusters and learning from the best, and the concepts and methodologies developed are in line with continuous improvement methodologies and the European Foundation for Quality Management (EFQM) (ECEI Process, 2019). Under the ECEI certification system, one of three management quality labels is awarded. The first, lowest level is a bronze label, followed by a silver label and a gold label, which is awarded to clusters that meet all the standards defined in the ECEI Management Quality Indicators (ECEI Criteria, 2013).

In order to receive the ECEI bronze label, a cluster shall notify the European Cluster Analysis Secretariat (ESCA) of its interest in participating in the process of pursuing excellence in cluster management and then allow one of ESCA's experts to conduct bench-marking tests. Each cluster that received a bronze label has been reviewed based on an interview with the cluster manager conducted by an ESCA benchmarking expert (ECEI Bronze, 2019).

The ECEI silver label is already a management quality label that confirms the successful implementation of the improvement processes initiated as a result of the benchmark test of the bronze label (ECEI Criteria, 2013).

The ECEI gold label is awarded to clusters that display highly advanced management as determined by an audit conducted by an ESCA expert and are committed to further improvement of their organizational structures and procedural evolution in order to achieve even higher performance. To become a gold label awardee, cluster management organizations must meet all specified “levels of excellence” in the categories analyzed (ECEI Gold, 2019).

The research subject of this paper are the opinions collected on the Internet about clusters and in-cluster businesses. The study randomly selected a total of 1,200 businesses, which were represented by 14,376 opinions and 24 clusters.

For the area of inquiry thus presented, we can formulate the following main objective:

CG: Evaluation of management quality determined by analyzing the sentiment of online opinions of cluster and in-cluster business customers with the level of cluster management quality measured by the ECEI label.

The main objective will be attained through the implementation of detailed objectives:

C1: Identification and evaluation of Internet sources of opinions for the purpose of analyzing the sentiment of Internet opinions of customers of clusters and in-cluster businesses.

In order to accomplish them, the paper focuses on answering the following research questions:

PB1. How to identify and evaluate sets of online opinion sources useful in analyzing the sentiment of clusters and in-cluster businesses?

In the light of this study, there is a significant disparity between the level of customer opinion on companies and the level of label they hold. The level of sentiment for businesses operating in silver-labeled clusters was 0.332 stars higher (on a scale of 1–5) than for businesses operating in bronze-labeled clusters. Noticeably, there was a lower disproportion between the level of sentiment for businesses in gold- and silver-labeled clusters. It amounted to 0.062 stars (on a scale of 1–5). This may be due to the methodology of awarding the ECEI. In order to award the bronze label, it is sufficient to declare interest in participating in the process of pursuing excellence in cluster management and to allow testing by an expert. The label is awarded regardless of the result. In the case of silver and gold labels, its award is a mark of management quality, which confirms the implementation of a number of processes that improve management.

It is important to notice that there is a disproportion between the level of customer opinion on companies (cluster members) and the level of the label awarded to clusters, as this makes it possible to measure the effectiveness of the cluster's subsequent action. The ECEI evaluation system does not assume such a possibility, which hinders continuous tracking of the effectiveness of improvement actions taken by the cluster. Sentiment analysis is a complementary tool to the ECEI rating system, and ongoing monitoring of the sentiment level of customer opinions about companies (cluster members) in relation to the label level provides an opportunity to assess the effectiveness of actions taken by the cluster to improve management processes.

The cluster business opinion frequency rate is another indicator estimated in the study. There is an increase in its level as the level of the ECEI labels increases by 3.39% between the bronze and silver labels and by 11.28% between the silver and gold labels. Such a significant shift for the gold label may be related to greater recognition of in-cluster businesses with this label, longer market presence, and greater awareness of how important it is for a company to be present online.

In conclusion, the study demonstrated the relevance of the addressed issue and the presented research results allowed for completion of the defined detailed objectives and for answering the set research questions. The main objective of the paper was achieved, that is, to compare the management quality assessment based on the analysis of the Internet sentiment of customers of clusters and in-cluster businesses with the level of cluster management quality measured by the ECEI label. The hypothesis that cluster management quality positively influences the increase in customer opinion sentiment about businesses – cluster members – was also confirmed.

As a result of the above, it appears that sentiment analysis may be applied as a complementary tool in measuring the effectiveness and efficiency of various business activities reflected in the reactions of peer online communities.