Stereophotogrammetry for 2-D building deformation monitoring using Kalman Filter

With increase in the rate of construction of skyscrapers, high rise buildings and other elevated structures in order to house the increasing population within the built environment, building deformation studies to mitigate the sudden failure of these structures have become inevitable (Ajibade, 1997; Olagunju et al., 2013; Wan Aziz et al., 2005). Deformation studies of such large structures are required in order to ensure that the structures are exhibiting safe deformation behaviour. Deformation refers to the changes a deformable body undergoes in its shape, dimension and position (Wan Aziz et al., 2005). As buildings get deformed due to age and wearing out of its constituent materials, their load-bearing capacity alters and this influences their dynamic response in terms of changes in shape, dimensions and strength. Over the years, several techniques have been utilized for monitoring the rate of deformation of structures. These techniques involve the use of either or a combination of geodetic surveys (Erol et al., 2004), photogrammetric measurements (Fraser and Gruendic, 1985; Fryer et al., 2007), satellite observations from GNSS receivers (Wan, 2002) and geotechnical/structural measurements (Fechteler and Eisert, 2009; Nwadialor, 2001; US Army Corps of Engineers, 2002).

The photogrammetric method has been found to be fast, accurate, cost effective and reliable (Mills and Barber, 2004; Wang et al., 2010) and this has popularized its use for deformation analysis. This method has gained prominence due to its capability for accurate and fast non-contact measurement of solid bodies (Luhmann et al., 2007; Maas and Hampel, 2006). The photogrammetric method of deformation analysis is entirely based on the principle of close-range photogrammetry (CRP) and as such, observations are made from accurately coordinated control points that are located at distance not exceeding 100 m from the observed structure.

One advantage of the photogrammetric method is the ease and speed of data acquisition per observational epoch, thereby reducing the cost and time required in data gathering for deformation studies (Albert et al., 2002; Lee et al., 2006). Large volume of data relating to the object to be studied is collected with just a snap of the camera. While the use of Terrestrial Laser Scanners (TLS) that is usually classified as being under geotechnical measurements also offers similar advantage, digital photographs have lower space requirements than TLS outputs. As such, due to manageable storage size, the output from CRP is easier to process with medium space personal computers (PC's) unlike their TLS counterpart. Consequently, the capability of CRP to produce dense point clouds similar to the output from TLS makes it a cheaper alternative to be considered in applications that require 3D position of points (Mokroš et al., 2018; Rüther et al., 2012).

Earlier works on the use of photogrammetric measurements for building deformation analysis have been done using photogrammetric software to generate 3D models of buildings and analysing positional differences between photo-generated coordinates and established ground positions over-time. Wang et al. (2010) used Photo Modeler Scanner Version 6 and Vision Metrology System (VMS) software to produce 3D models of membrane structures. Thereafter, the study determined deformations by comparing the differences in linear measurements between points on the photogrammetric model and physical measurements taken as control points (a method known as check measurements). Similar method was utilized by Fawzy (2015) to determine deformations in a 41 m tall building in Egypt using the Photo modeler software.

The suitability of stereo-photogrammetry to measure lateral and vertical deformation of a reinforced concrete column under dynamic loading had been studied by Alemdar et al. (2011) and found to be suitable for tracking lateral and vertical displacements. Chounta and Ioannidis (2012) used 3D photo-grammetric coordinates obtained from space resection and intersection to calculate point displacements on a concrete beam during a loading test. Images were taken in load free state and in every loading epoch until the beam reaches its breaking curve. In every epoch, an automatic target localization algorithm, based on a combination of normalized cross correlation and least squares matching was applied in order to measure image point coordinates’ pre-signed points on the beam. Also, Lee and Han (2018) utilized the 3D coordinates of targets on a bridge obtained from photogrammetric measurements to determine the deformation on a bridge by analysing the differences between coordinates at different epochs with specified baseline coordinates. The study used a photogrammetric board to determine the interior and exterior orientation parameters. Baseline coordinates for the deformation analysis was obtained from TLS data. In order to ensure consistency in the points whose differences were to be analysed, a target-matching of sequence of photos was developed.

Invariably, we infer from previous studies that the photo-grammetric technique has been very effective for static deformation monitoring. However, most of the current engineering applications require monitoring of movement behaviours; and as such, a kinematic deformation model that determines displacements, velocities, acceleration as well as time variation is required. The most frequently utilized kinematic model is the Kalman filter. The Kalman filter (KF) algorithm has been used extensively in building deformation studies where GPS or other ground survey techniques have been used for field measurement, after which subsequent analysis of movements is done using KF (Ehigiator-Irughe and Ehigiator, 2013; Gulal, 2013). Nevertheless, its use in photogrammetric deformation analysis is not common. The application of KF in photogrammetry has been limited to localization positioning where the KF algorithm is used to model the navigation path of a robot in robotic sciences (Jung, 2004). This research therefore integrates the use of 2D photogrammetric transformation models to transform photo coordinates to ground coordinates, thereafter, utilizing the KF algorithm to detect possible movements in the building. Stereo photograph of a building was taken using a digital camera mounted on a tripod stand in-order to analyse the possibility of 2D movements within the building. This research has focused on both vertical and lateral tilts of the structure studied. The choice of stereo-photography for 2D analysis in this study is to minimize the effect of parallax in the resulting 3D image of the captured scene.

Kalman filtering, also called the linear quadratic estimation (LQE), uses a series of measurements observed over time, containing statistical noise and other inaccuracies to produce estimates of unknown variables known as the state vector in a recursive least squares manner (Ince and Sahin, 2000). The intention of a KF as other kinematic models is to find the suitable description of point movements by time function without regarding the potential relationship to causative forces (Ehigiator-Irughe and Ehigiator, 2013). The KF predicts the present state vector of an object based on state vector information of known motion parameters at period t − 1 and measurements taken at period t. The state vector of motion parameters consists of positions, motion and acceleration variables; such that the motion and acceleration parameters are the first and second derivatives of the positions measured with respect to time. The Kalman-filtering algorithm for detection of possible movements in structures can be mathematically represented in matrix notations as given in equation (1): (1) Xt=ftXt−1+Btut+wt {X_t} = {f_t}{X_{t - 1}} + {B_t}{u_t} + {w_t} where:

X_t is the state vector containing the terms of interest for the system (e.g., position, velocity, heading) at time t;

Assuming that all conditions are the same, we can take B_t as unity. Therefore, equation (1) can be re-written as equation (2) (2) Xt=ftXt−1+ut+wt {X_t} = {f_t}{X_{t - 1}} + {u_t} + {w_t} Therefore, the vector of unknowns consists of positions, velocity and acceleration (only velocity is considered in this study). These can be calculated with a minimum of three cycles of measurements at different times.

The velocity is represented as given in equation (3): (3) VXtt=Xt−Xt−1Δtt,t−1 V_{{X_t}}^t = {{{X_t} - {X_{t - 1}}} \over {\Delta {t_{t,t - 1}}}}

The area of study is School of Environmental Technology Lecture Theatre of the Federal University of Technology Minna, Gidan Kwano campus (Figure 1). It is located along Bida – Minna express road, Bosso Local Government, Niger State, Nigeria. The structure has been in use for over 10 years and hosts approximately 500 persons at full capacity.

Figure 1

The study presents an empirical research on the use of photogrammetric-derived coordinates for performing 2D deformation analysis of a building. The materials used for the study are as presented in Table 1.

First, conventional ground survey technique is used to determine the coordinates of two selected ground control points (GCPs) from where the overlapping photographs were taken. The points were identified as SETA01J and SETA02J. In-order to achieve the desired results, the point location were fixed such that there was a complete coverage of the building from both control points. The Leica Total station was used to coordinate the GCPs and a sketch of the traverse observation is shown in Figure 2. Figure 3 depicts the location of the pillars viz. the building for the acquisition of photographs. Thereafter, suitable locations on the object are marked to be used as Photo Control Points (PCPs) or targets, which would be easily identifiable on the photograph. The object space coordinates of the chosen PCPs would first be determined by 2-points intersection. This first determination of the PCP coordinates by 2-point intersection would serve as baseline values for the deformation analysis.

Figure 2

Figure 3

In this study, four bulbs marked and labelled as A, B, C and D located about the middle of the building were used as PCPs. The choice of selection of the points was to ensure that the points picked was within the load bearing portion of the building. The ground position was not chosen because such points apart from being susceptible to relative/differential movements could also not appropriately reveal the vertical movements in the building. Therefore, with the chosen points, the absolute movement of the building was determined.

After the initial coordinates of the PCPs are determined by the 2-point intersection method, overlapping photographs of the building are taken from the GCPs at each observational epoch. The digital camera used for image acquisition was calibrated using a checkerboard. The stereo-pair is then analysed with MATLab for extraction of pixel coordinates. A polynomial transformation function is then developed from the relationship between the known coordinates of the PCPs and the pixel coordinates (Odumosu and Ajayi, 2014). This polynomial function was used to convert the pixel coordinates obtained per epoch into real world coordinates that were finally used for the building deformation analysis. The accuracy of the transformation should be evaluated using the covariance matrix of parameters given in equation (4). (4) Qx=σ^o2(APAT)−1 {Q_x} = \widehat \sigma _o^2{\left( {AP{A^T}} \right)^{ - 1}} where: Q_x is the covariance matrix, σ^o2 \widehat \sigma _o^2 is a measure of goodness of fit of observations (standard error), A is the design matrix and P is the a-priori weight of observations.

A summary of the methods employed in this study are as listed below:

Select suitable ground stations to serve as Ground Control Points (GCPs) from where overlapping images of the building could be taken. Determine the position of these points using ground surveys.

Presented below are the results obtained from the different work phases. Table 2 shows the summary of accuracy achieved during the traverse operation conducted to determine the coordinates of the GCPs. As seen in the table, a linear accuracy of 1:3000 was obtained, which indicates a third order accuracy traverse. This implies a possibility of a linear displacement of 1 m in every 3000 m.

Also, the transformation parameters determined for converting pixel to object coordinates are presented in Table 3. The parameters were used in a polynomial transform to determine ground coordinates from pixel coordinates. It should be noted that although a total of 10 PCPs are located across each photograph, four of the points (not used for deformation analysis) were left out for validation. The computed standard error ( σ^o {\widehat \sigma _o} ) obtained from the four validation points after applying the determined parameters presented in Table 3 was found to be ±0.00005329 m. This implies that the derived transformation parameters are correct and computed coordinates useful for sub-millimeter analysis.

The computed coordinates of the four PCPs used for the deformation analysis obtained from the transformation of pixel coordinates to object space coordinates at the three observation epochs are presented in comparison with the observed ground coordinates of same points in Table 4. Very close similarity is seen in the field measurements and the photogrammetric derived coordinates. The correlation noticed in both datasets illustrates a measure of the precision of the transformation method used for converting pixel coordinates to object space coordinates. This high level of correlation is expected between both datasets being a true indication of reality. Furthermore, the correlation also indicates that transformation method used in the study is comparable to the space resection and intersection methods. Each observation epoch consists of a set of stereo-pair observations taken to the building in a day. The epochs were taken at two months interval to each other. Differences between the successive epochs and the baseline coordinates are presented in Tables 5–7. The difference in coordinates noticed at all the four PCPs range from −0.7 to + 0.7 m. This indicates a centimetre-level coordinate difference and suggests near stability of the structure.

The result of the KF analysis is presented in Table 8. The observed differences in coordinates between successive observation epochs and the baseline observation were used as input data to determine the initial and subsequent state vectors of the building. The results from the KF analysis indicate that the building is stable over a period of four months within which the study was carried out. This is seen by very small and Ins. Mov.s at all the four PCPs. Table 9 shows the detected motion (in terms lateral and vertical velocity) at the points A–D during the 3 observation epochs.

Table 2 indicates an acceptable level of accuracy (3^rd order) in the initial traverse with which the coordinates of the GCPs were determined. Similarly, reliable value of standard error obtained from the variance matrix of parameters (±0.00005329 m) of the polynomial transformation indicates high level of accuracy of the polynomial transform in determining the ground equivalent of pixel coordinates of the PCPs. This implies that large discrepancies between ground and photo-generated coordinates would actually reflect either poor pixel coordinate extraction and/or large movements in the building.

The KF analysis (Tables 8 and 9) showing horizontal and vertical motions at the check points for each of the observational epochs indicate that at the first epoch, all check points of the structure are stable, while at the second and third epoch (bi-monthly), Ins. Mov. is noticed at all the check points. This result is consistent with the findings of Ehigiator-Irughe and Ehigiator (2013) where a maximum velocity of 0.04 m/year is observed at one of the monitored points of an overhead tank (over a period of one year). PCP ‘C’ is the most stable point amongst all the observed points. A possible reason for such great stability is the presence of a vertical column along its path. These results are graphically substantiated as shown in Figure 4. At PCP ‘A’ (first check point) Ins. Mov. is noticed in the second and third epoch of observation in both the horizontal and vertical directions. Similar trend but of greater magnitude is noticed at the second check point. The horizontal movement though insignificant is more at PCP ‘B’.

Figure 4

Smallest movements were noticed at PCP ‘C’ although a spurious spike is observed in the horizontal movement at that point in the third epoch. The reason for this sudden deviation from previous trend is uncertain and should be investigated by a loner observational period with a more consistent interval of observations. PCP ‘D’ again shows very minimal and Ins. Mov.s as the case at other points. The findings of the research are illustrated by charts presented in Figure 4.

Generally, all the movements are in the same direction (positive) with time except the horizontal movement at the third epoch for PCP ‘D’. This general trend indicates increasing deformation within the structure with continuous use of building over time. This trend is expected and justifies the need for routine maintenance of the structure as the building continues to age. Nevertheless, the exceptional trend noticed at PCP ‘D’ as seen in the horizontal motion in Figure 4d suggests that the deformations are elastic and not plastic. The result is in agreement with the result of Mustafin et al. (2017) that discovered elastic deformation in their research work and noted it as temporary change in shape, the modification that is self-reversing especially when the force causing it is removed, so that the object returns to its original shape. Structural members undergo elastic deformations due to dead load or super imposed live loads. Consequently, elastic deformations are load induced deformations that reverse themselves when such loads are removed. Since elastic deformations if unchecked may lead to cracks in buildings, further investigation is required around PCP ‘D’ to identify its cause and possibly reinforce that section of the building. By implication, it can be inferred that while emphasis need be placed on magnitude of movements in buildings, the direction of movement overtime is also a very useful indication of the type or nature of deformation such buildings are experiencing.

Photogrammetric derived coordinates obtained by the application of a polynomial transform to convert pixel coordinates of a stereo-pair to object coordinates have been analysed by the KF algorithm to detect movements of a building. The results obtained indicate the viability of the use of stereo photogrammetry technique for detecting movements in any engineering structure. The School of Environmental Lecture Theatre (study area) has been studied via this procedure and discovered to be generally stable with Ins. Mov.s ranging from 0.0000001–0.000007 m/epoch (an epoch is a two month interval) at four identified check points (PCPs) with greater movements in the horizontal direction. The trend of movement indicates that the building, though stable, is experiencing salient movements in response to usage and/or years of construction. Furthermore, the graphical plot of the movement reveal the presence of an elastic deformation at one of the check points (PCP ‘D’).

Further studies using similar technique is therefore recommended but to be conducted at regular intervals (bi-monthly) over a period of about five (5) years so as to effectively conclude on the monthly/annual rate of rise/sink and lateral deformation of the building. Such information will inform the university administrators on the appropriate duration for building reinforcements, points to be strengthened and also number of years the structure can withstand prior to a possible collapse.