Innovative approaches to productivity monitoring: Integrating work sampling and electronic performance monitoring

This research applies the DSR methodology (Nunamaker and Purdin 1990; Peffers et al. 2007) to demonstrate and evaluate the development of a mixed method using electronic monitoring and WS for labour rating assessment. Figure 1 presents the research method developed based on the DSR methodology.

Fig. 1:

Research method.

Existing research on workforce location-based performance monitoring has primarily focused only on geolocation and activity information without being able to do a deep quantitative and qualitative assessment of productivity and its correlated impact factors. On the contrary, a few studies applied modelling process techniques to qualitative and quantitative productivity measurements. Also, manually collecting data by observations can be time-consuming, costly and labour-intensive.

Hence, this study contributes to knowledge by designing/developing, demonstrating and evaluating an integrated method to assess workers’ productivity through (a) using smartphones, which is a very available technology nowadays, (b) for continuous monitoring of labour rating, calibrated by (c) a WS methodology of human observation, which can (d) diagnose productivity labour rating and impact factors. Consequently, this work aims to provide an integrated method using new technologies and a traditional modelling process technique to assess and empower the continuous improvement of the workforce’s productivity cycling.

The experiment was conducted at a pipe shop in the State of São Paulo, Brazil. The plant has a total area of 73,117.00 m² with a constructed area of 16,492.00 m² and stands out in the manufacture of pipes for numerous Brazilian heavy construction industry projects. The experiment was carried out at the service fronts of the 10 welders under analysis during their usual activities on regular working days, see Figure 2. During the assessment period, the welders conducted activities involving welding low-carbon alloy steel pipes using Tungsten inert gas (TIG) and flux-cored arc welding methods (FCAW) techniques. This study was conducted in compliance with ethical guidelines for research involving human subjects. All participating welders were informed about the nature of the study and provided written consent for their involvement. Worker privacy was maintained by anonymising all collected data, and participants were given the option to withdraw from the study at any time without consequence.

Fig. 2:

Welders use smartphones, and researchers conduct the WS. WS, work sampling.

This work evaluates the effectiveness of a methodology to monitor and quantify workers’ productive and unproductive activities. In this sense, considering that the worker must be close enough to handle tools and materials to perform a certain task (Navon and Goldschmidt 2003a, 2003b; Navon 2005). The work area was divided into three distinct zones based on their relevance to the welding process: Production zone (PZ), the primary area where welding activities occur; Idle zone (IZ), an area where workers are present but not actively engaged in production and Mixed zone (MZ), transition areas between PZ and IZ. These zones were defined based on GPS coordinates and mapped to specific activities observed during the WS process. Figure 3 presents the concept of productive zones based on workforce occupation.

Fig. 3:

Productive zones are a concept related to workers’ occupations. IZ, idle zone; MZ, mixed zone; PZ, production zone.

To achieve autonomous data collection using electronic devices, smartphones with proprietary embedded software were used, with subsequent data collection and processing by web software, as well as developed proprietary technology for the experiment. Preliminarily, the work areas were geographically registered in the system. In this way, the zone of productive occupation was identified. Workers must carry the device throughout the working day so that their geographical position can be detected every minute. The developed app uses GPS, Wi-Fi and mobile networks to determine the location of smartphones. According to the system response, the latitude and longitude data were captured for the experiment in a timeslot-based manner with the highest accuracy. At the beginning and end of the welding of a joint, the employee should take a picture of the barcode that identifies the spool. Through the Wi-Fi network and/or 3G, the data stored on the device are sent to the web software for data storage and analysis. Figure 4 illustrates the electronic monitoring process applied. The process involves (1) workers carrying smartphones with proprietary software, (2) continuous GPS location tracking, (3) barcode scanning of spools at the start and end of welding tasks, and (4) data transmission to web software for storage and analysis. This system enables near real-time productivity monitoring and calculation of the electronic factor of the production zone (EPZ).

Fig. 4:

Schematic representation flow of the EPM process conducted. EPM, electronic performance monitoring.

These tools enabled by the proprietary web software allowed the development of an internal method that indicates the percentage of occupation of the welders inside and outside the PZs previously registered with the export of these points within a precision of 10 m. The following is the ordered sequence of steps that has been established to perform this procedure and which represents the process applied:

Identify and remove the points related to days with single-area sampling;

Thus, it was established that the EPZ percentage is the ratio between the sums of the number of points contained in the area recorded as productive in relation to the total points collected. In the same way, to obtain the electronic factor of the idle zone (EIZ), the non-productive zone occupation consists of the sum of the points counted outside of the productive zone concerning the total ratio of points. Finally, as described in the methodology of the sampling proportion, the averages were validated through the Monte Carlo analysis, and based on the number of points collected in this sample, the confidence limit of the results of the electronic methodology was delimited.

In the WS methodology, the data are collected through human observation and randomly performed and stratified according to the productive state, whether performing an activity (productive) or not (non-productive) (Sink 1985; Meyers and Stewart 2002; Adrian 2004; Groover 2007; Niebel and Freivalds 2013). The number of individual observations in each variable is transformed into a percentage value based on the ratio of the total observations. According to Adrian (2004), the sum of the percentage values of the activities considered productive determines the LRF (Adrian 2004). Therefore, the sum of the percentage values of the activities considered non-productive determines the idleness rating factor (IRF), according to Eqs. (1) and (2) (Adrian 2004).1LRF= Total observations of productive state Total observations 2IRF = Total observations of non − productive state Total observations

To ensure the validity and reliability of the WS observations, two trained observers were employed in addition to the primary researcher. All observers underwent a comprehensive training programme to ensure consistent understanding and application of the observation protocol. A detailed observation guide was developed, clearly defining each activity category and providing classification guidelines. Observations were conducted according to a randomised schedule to prevent systematic bias. Inter-observer agreement was calculated using Cohen’s kappa coefficient, with a result of 0.85, exceeding the recommended threshold of 0.80 for qualitative research (Finkler et al. 1993). This high level of agreement demonstrates the consistency and reliability of the observational data collected.

This WS methodology is noteworthy because it relies solely on the count of observations without requiring continuous time measurement (Sink 1985; Meyers and Stewart 2002; Adrian 2004; Groover 2007; Niebel and Freivalds 2013). Increasing the recorded observations will directly correlate with an increase in confidence and a decrease in the error limit (Adrian 2004). However, irrespective of the sample size, there can always be an inherent uncertainty in the results (Adrian 2004). The uncertainty rate in modelling is quantified using three statistical concepts: confidence limits, error limits and category proportions (Adrian 2004); see Table 1.

The data collection procedure aimed to detect workers’ activities that were considered productive or non-productive. Observations are taken on a timely basis, whether the timing is registered, but assuring action marking of the welder is performed at the very moment of observation. During the data collection of this experiment, the observation of the activity of each welder occurred in a time not less than 5 min for the same welder, where at random during the working day, data of the 10 welders who participated in the experiment were collected.

The simulation using the Monte Carlo technique is utilised to develop the probability distribution function (PDF) of LRF and IRF. The software @risk 7.5 (Palisade Company, LLC, Ithaca, NY 14850, United States) was employed for this purpose. The PDF obtained corresponds to the sum of the functions, representing each event’s behaviour as productive or non-productive considering the population of all actions observed. The simulation process basically follows the steps (Calvetti and Ferreira 2018):

Clustering the observation data collected in MS Excel worksheets according to the classification within productive state and non-productive state;

Finally, based on Adrian (2004), the sampling observation proportion model was applied and compared to validate the PDFs Monte Carlo modelling (Adrian 2004). This analysis compares the average values of the functions and their standard variations for deviations/coefficients to the calculated values. It also compares the error limits obtained using Table 1 to assess the accuracy of the results.

To compare the adhesion of the results obtained through electronic methodology and human observation, each method’s averages and probability density function curves are analysed individually and in relation to each other. Likewise, the impact factors identified in the rating factors and their occurrence zones were evaluated.

In addition, based on the data obtained, a methodology of calibration for the electronic monitoring was developed, which incorporates the possibility of the occurrence of idleness within the established PZ. Finally, an integrated analysis is made to achieve the best workforce productivity assessment based on both methodologies.

The electronic methodology generated 10,544 points regarding the positioning of the 10 welders under analysis, identifying that 6,092 of these points occurred within the PZ. Thus, the EPZ reached 57.78%; consequently, the EIZ was 42.22%. Applying the confidence limit methodology proposed by Adrian (2004) within the same parameter selected for the sampling allows it to assess the limit error in relation to the electronic sample obtained (Adrian 2004).

Thus, for the sample relative to 10,544 observations and interpolating the values from Table 1, the limit error of approximately 0.78%, within a 95% confidence limit, is reached for the proportion category 50%. These results make it clear that the higher number of points sampled in the electronic method guarantees a lower limit error than that produced by human observation sampling. Considering a limit error of 2.41% obtained in the human observation method and 0.78% in the electronic method, a difference of the order of 309% is observed. This fact is mainly due to the following points: time of data collection; human observation leads to the mobilisation of a team of researchers in front of the service and the electronic method is autonomous, allowing the uninterrupted collection of data.

The PDF of the EPZ obtained by Monte Carlo simulation from the sample data of each of the 10 welders considered as independent variables, where an average of 57.649% is obtained, accounting for a deviation of 2.228, max. of 80.581, min. of 55.512 and a variation coefficient of 0.0386; see Figure 5a. This value is very close to that obtained by the total sampling, corresponding to 57.78%, which validates the function. The mean value of the PDF of the EIZ was 42.336%, with a standard deviation of 1.675, a maximum of 45.677, a minimum of 31.623 and a coefficient of variation of 0.0396, which is very close to the value obtained by the total sampling of 42.22%, which validates the function determined, see Figure 5b.

Fig. 5:

PDF: (a) EPZ; (b) EIZ. EIZ, electronic factor of the idle zone; EPZ, electronic factor of the production zone.

The information gathered from the welders’ work was organised based on their activities, amounting to 3,577 recorded observations. Depending on the welding process involved, these activities were classified as productive or non-productive. Consequently, Table 2 showcases the number of observations in each action correlated with the productive state, and Table 3 shows the observation regarding the non-productive state. The dataset has been used and presented in early research and will be used to compare the WS method and the electronic one, serving as a base for EPM calibration (Calvetti and Ferreira 2018). The productive activities listed in Table 2 were determined through direct observation of the welding processes in the study environment. While these activities align with common practices outlined in standards such as ISO 3834 series, the list is not exhaustive and may vary depending on specific project requirements and welding techniques employed. At the same time, the non-production activities presented in Table 3 were identified through a systematic WS process. This process involved an initial observation period, iterative refinement of activity categories and expert validation by industry professionals. To manage experimental and methodological bias, several measures were implemented, including rigorous observer training, randomised observation schedules, multiple observers, blind recording techniques and regular inter-observer reliability checks. A continuous feedback loop was established to ensure consistent activity classification, and data triangulation was employed where possible to verify observational accuracy. These methodological safeguards were crucial in ensuring the reliability and validity of the identified activities, providing a robust foundation for subsequent analysis of workforce productivity.

The LRF density curve was generated via simulation and is the cumulative result of functions connected to the productive state. The probability distribution function (PDF) computed average is 0.5529, with a standard deviation of 0.0525, Figure 6a (Calvetti and Ferreira 2018). The PDF reaches a max. of 0.7942 and dips to a 0.3262 minimum, with a variation coefficient recorded at 0.09 (Calvetti and Ferreira 2018). In the same way, the density curve of the IRF is a result of the functions simulated and associated with the non-productive state. The value average of the IRF-PDF is 0.4549, with a deviation of 0.1035 (Calvetti and Ferreira 2018). The IRF-PDF reaches a max. of 0.9692 and a min. of 0.1785, with a variation coefficient of 0.23, Figure 6b (Calvetti and Ferreira 2018).

Fig. 6:

The PDF: (a) LRF-PDF (Calvetti and Ferreira 2018); (b) IRF-PDF (Calvetti and Ferreira 2018). IRF, idleness rating factor; LRF, labour rating factor.

The LRF and IRF are calculated by comparing the ratio of absolute observations in each state to the total observations collected (Adrian 2004). Therefore, considering the sample of 3,577 observations, where 1,986 were clustered as productive state and 1,591 as non-productive state, the LRF was found to be 55.52%, and the IRF was 44.48% (Calvetti and Ferreira 2018).

Additionally, it can determine the relative error limits regarding the sample size within a given confidence interval. Based on Table 1, a category of 50% proportion was selected within a 95% confidence limit; subsequently, given the number of observations of 3,577, the error of 2.41% was calculated (Calvetti and Ferreira 2018). Therefore, the LRF found is considered in the range of 53.11% and 57.93% (±2.41%) (Calvetti and Ferreira 2018). At the same time, the LRF obtained by the PDF simulation of 55.30%, with a standard deviation of 5.25%, can be found in the range of 50.05% and 60.55% (Calvetti and Ferreira 2018). The Monte Carlo simulation of the LRF exhibits expected variability due to the sum of its constituent random variables, and the resulting PDF accurately represents the LRF’s behaviour, as confirmed by its close alignment with sampling chart results.

Analogously to the LRF, the proportion of IRF determined by the sampling observations is 44.48% (Calvetti and Ferreira 2018). Using the same category and confidence limit, the IRF is considered in the range of 42.07% and 46.89% (Calvetti and Ferreira 2018). Also, the IRF-PDF presented 45.49% as the mean value, with 10.35% of the standard deviation (Calvetti and Ferreira 2018). In this case, the IRF-PDF indicates results ranging from 35.14% to 55.84% (±10.35%) (Calvetti and Ferreira 2018). The IRF results show a greater dispersion in its PDF than the LRF due to the complexity and diversity of variables found in the sample, leading to increased dispersion in the Monte Carlo simulation (Calvetti and Ferreira 2018). However, the methodology can still be considered suitable, where improving activity modelling and increasing observations can reduce this dispersion found (Calvetti and Ferreira 2018).

A study using the WS method to evaluate tiling crew activity in an industrial building measures a level of active production of 65.43% and non-productive time of 34.57% (Shehata and El-Gohary 2011). Using the WS method, the active classification rate for prefabricated concrete elements was 42.00% direct work rate at 42.0%, 27.20% supporting activities and 30.80% non-productive activities (Lindhard 2023). Different case studies measured productive state rates for building renovation as (Wandahl et al. 2023): 25.03% carpenter work; 21.62% diverse renovation works, such as replacement of balconies, kitchens and bathrooms, replacing windows, adding insulation and putting up drywall; 34.57% works of new facades, new roofs and completely new installations; 38.96% new works on brick facades, roof, windows, additional and new insulations and 31.19% new facades, balcony, kitchen, bathroom and installations (Wandahl et al. 2023). A study of Danish renovation projects of a building measured a rate of 29% of productive actions for the crews in analysis (Pérez et al. 2022). For plumbing activities, a productivity rate of nearly 38% was found (Nassri et al. 2023). The different results using the WS method from distinguished use cases can indicate reasonableness in observing more productivity occupancy in works with more industrial characteristics.

Considering the results of 55.52% for the LRF and 57.78% for the EPZ, an absolute difference of 2.26% was observed between the results of the different methods. The difference obtained is low, considering that the electronic methodology does not detect idleness in the PZ. However, in this study, this fact may have influenced this outcome once workers knew they were being monitored. However, it is worth mentioning that idleness in the PZ can be measured using the human observation method, allowing it to be considered when implementing the electronic method. At the same time, due to the IRF in a simple perspective being the difference between the total and the LRF and appearing with high dispersion in the PDF simulation, it is more valuable to calibrate the electronic method correlating only EPZ and LRF.

As such, the difference between the methods of human observation and the electronic methodology is given by the limitation of the electronic detection of idleness occurring within the PZ. Likewise, based on the analyses of the observations, it is possible to characterise the actions identified according to their place of occurrence and their proportional percentage.

Productive state:

Welding, 29.91% (PZ);

Non-productive state:

Walking, 18.62% (IZ);

In this case, it is observed that the characteristic productive actions, which determine the LRF, occur in the vast majority, 91.39%, according to Eq. (3), within the PZ. Except for the following activities: ‘Closing activities, organising and cleaning the workplace’ and ‘Cleaning the work area’. The activities are part of the PZ and part of the IZ, that is, in the MZ, as previously mentioned.

The analysis indicates that these two activities have a low impact on the average LRF output. It is also observed that the action ‘Interruption of activities due to the tubes rigging’ is idle and occurs in the PZ. Just like actions 2 and 3 above, the actions: ‘No joint released to start welding’, ‘Waiting for crane operation’, ‘Awaiting quality inspection’, ‘Human physical conditions’ and ‘No welding machine or defective machine’ occur in the MZ. The actions that occur only in the IZ are: ‘Walking’ and ‘Management, health and safety meetings’.

To consolidate the registered data of the actions by zone of occurrence, the following results are obtained: 51.08% in the PZ, 22.87% in the IZ and 26.06% in the MZ. Based on the result of 57.78% of the EPZ, compared to 55.52% of the LRF, as well as 91.39% of the occupancy in the PZ, it can be inferred that employees within the PZ tend to occupy themselves. This fact can be justified by the occurrence of superior supervision and of their peers within the front of service because greater idleness is identified in places outside the PZ.3Ocupation on the PZ=1− action 2+ action 3 LRF ×100 where:

Action 2 = Closing activities, organising and cleaning the workplace;

Action 3 = Cleaning the work area;

LRF = 55.52 total of the LRF.

As shown in Figure 7, based on the result of 57.78% of the EPZ, compared to 55.52% of the LRF, there are differences between the probability distribution functions. As discussed above, it mainly arises from interpreting the labour rating in the MZ.

Fig. 7:

Probability distribution functions of EPZ and LRF comparison. EPZ, electronic factor of the production zone; LRF, labour rating factor.

To compare the behaviour between the curves obtained by both methodologies, several functions derived from the PDF of the EPZ were generated, Figure 8. The plotted curve of +2% covers the upper limit until 82.58%. On the contrary, to cover the bottom limit until 35.50%, a plotted curve of -20.00% is needed. Based on the intersection of the curves, the correlation limit is determined to be between 82.58% and 58.00%, and the correlation of the electronic value related to the human observation method is ±2.00%. In this hypothesis, a determined EPZ of 60.00% is calibrated as an LRF of 58.00% until 62.00%. For instance, the calibration abacus shows respectively for values between 54.50% and 58.00% a correlation limit of ±4.00%, 52.00% until 54.50% correlation ±6.00%; 50.00% until 52.00% correlation ±8.00%; 47.50% until 50.00% correlation ±10.00%; 45.50% until 47.50% correlation ±12.00%; 40.50% until 45.50% correlation ±17.00% and 35.50% until 40.50% correlation ±22.00%.

Fig. 8:

Calibration abacus EPZ to LRF. EPZ, electronic factor of the production zone; LRF, labour rating factor.

According to Adrian (2004), it is expected that, in construction projects, on a workday of 8 h, only 4 h is productivity (Adrian 2004). As a result, it is anticipated to be at least 50% of productivity, indicating an accuracy of 84%–96% for the model.