An Automatic Inspection Method for the Fracture Conditions of Anisotropic Conductive Film in the TFT-LCD Assembly Process

Abstract

In this study an automatic optical inspection system is presented to evaluate the fracture and deformation status of conducting particles of anisotropic conductive film in the TFT-LCD assembly process. The amount of deformation and quantity of conducting particles in the test pattern can be automatically evaluated by image analysis. A specific operation is carried out in the image processing method, and the calculation of the image gradient operator is used to produce a preferable contrast between the processed particle image and the background. The thinning processing method is applied for information reduction and information creation. An amount of samples are taken with a target template for synchronous multiple-comparison, and the optimal threshold of the binary image is obtained. This study utilizes the assistance of image processing technology to inspect the fracture conditions of anisotropic conductive film in the TFT-LCD assembly process. This system can decrease the defection rate of products, obtain over 90% recognition accuracy even in noisy environments, and will be verified in an automatic production line.

Keywords:

• anisotropic conductive film
• automatic optical inspection
• fracture
• image processing
• thinning processing method

NOMENCLATURE

f(x,y)	=	the gray level of image
∇f	=	the gradient of image f(x,y)
G x	=	the gradient value along x axis
G y	=	the gradient value along y axis
z 1 ∼ z 9	=	the gray-scale value of the pixel before Prewitt mask operation
g x	=	the new gray-scale value along x axis after the Prewitt mask operation
g y	=	the new gray-scale value along y axis after the Prewitt mask operation
g1 ∼ g4	=	the new gray-scale value after each Prewitt mask operation
f 1(x,y), f 2(x,y)	=	the gray-scale value of two images' corresponding pixel
σ0, σ1	=	the variances of part C 0 and C 1
σ W	=	within-class variance
σ B	=	between-class variance
στ	=	total variance
P i	=	the probability density function
ω0, ω1	=	is the area of this aggregation in histogram
μ0, μ1, μτ	=	the distance from the centroid of area of C 0, C 1, and total range in histogram to the ordinate axis
k*	=	the optimized threshold

1. INTRODUCTION OF ANISOTROPIC CONDUCTIVE FILM AND CONDUCTING PARTICLES

In the present TFT-LCD (Thin Film Transistor Liquid Crystal Display) panel process, IC (Integrated Circuit) flip chip packaging technology combined with anisotropic conductive film is mainly used. The function of anisotropic conductive film (ACF) is to provide the conductance of two joiners vertically, while there is an insulation effect in the horizontal direction. The anisotropic conductive film is mainly composed of conducting particles and binder. The driving IC, tape-carrier packages IC are fixed on the liquid crystal display panel (Lin et al. 2009; Maeda et al. 1999). Anisotropic conductive film is adhered to the terminals of the tape-carrier packages IC and PCB (Printed Circuit Board). The conducting particles of the anisotropic conductive film are fractured and deformed by controlling the temperature and the press fitting of pressure to form the conduction circuit. Anisotropic conductive film is mainly composed of conducting particles and insulation rubber. It also has conduction, insulation, and adherence functions. When it is heat sealed, the conductance of conducting particles is formed in the vertical direction of the thickness of film, but a monodisperse distribution is among the conducting particles. Since there is still sufficient space, the film direction is insulation, as shown in Figure 1 a. The distribution diagram of conducting particles inside the anisotropic conductive film and its structural diagram are shown in Figure 1 b.

Figure 1 (a) Distribution diagram of conducting particles inside sealing ACF. (b) Diagram of the structure of conducting particles inside the ACF and particle distribution inside the film (color figure available online).

The conduction method of anisotropic conductive film utilizes the indium tin oxide pole of conducting particles on a patch panel and pad of the IC for conducting electric apparatuses.

The indium tin oxide pole used in liquid crystal display is a kind of transparent conductive film coated on the surface of glass substrate by vaporization. It has good conductivity, and is called indium tin oxide glass. The conduction characteristics are the most important requirements of the Chip On Glass product process. Besides the resistance value of indium tin oxide glass, the conductivity of the integrated system should be considered. The amount of deformation and quantity of conducting particles are very important, and are closely related to the setting of each parameter in the process(Lin et al. 2003). Higher process pressure often leads to a larger deformation of the conducting particles, namely, the particles would flatten. Thus, the contact area of the conduction part would be enlarged, and the contact resistance decreases. Contrarily, if the pressure is too low, the contact area would be insufficient, and the contact resistance would too high, resulting in poor conduction. Therefore, it is very important to evaluate the deformation conditions of the surface of anisotropic conductive film in the optomechatronics process.

Hsieh (2010) used an integrated procedure incorporating the data mining techniques to achieve the construction of yield loss model in the analysis of manufacturing process of liquid crystal display products. Chiang and Hsieh (2009) used the correlation operation between quality characteristics and applied the principal component analysis to eliminate the multiple co-linearity in the inspection of LCD color filter manufacturing. Two quality characteristics and four control factors with three levels were selected to optimize the thin-film sputtering process with the entropy measurement method. (Lin et al. 2007; Lim and Jeong 2007).

However, these approaches are not suitable for the automatic inspection of fracture characteristics of anisotropic conductive film. Because the test pattern of anisotropic conductive film is very complicated. The quantity, fracture state, and distribution state of sealed conducting particles would affect the conduction between the liquid crystal display panel and its ambient control circuit. It's difficult for analysis. In this article we proposed a thinning and pattern template matching method for the automatic optical inspection system in LCD and anisotropic conductive film assembly process to improve the conventional imaging technology. This study also carried out research and development designs for inspection technology with specific image processing method in relation to the test pattern of anisotropic conductive film conducting particles. In this system, it can track, position, and inspect images of conducting particles, and measure the fracture and deformation status by using the machine vision method.

2. TEMPLATE FOR SYNCHRONOUS MULTIPLE-COMPARISON

The diagrams of conducting particles inside the panel circuit are usually complicated, and there are many irregular segments and dots. Therefore, to increase the contrast between the background and target image, the image pretreatment for the diagram to be inspected is computed (Guerra and Villalobos 2001; Zheng et al. 2002; Lin et al. 2008).

The image pretreatment chiefly uses the calculation of image gradient operators to reduce the noise (Salembier et al. 1996; Lin 2007).

The gradient of image f(x,y) at position (x,y) is defined as vector ∇f:

where f(x,y) is the gray level of image (Figure 2 a). Gradient vector ∇f is the direction of maximum rate of change of f(x,y) at coordinates (x,y). G _x and G _y are the gradient value along x and y axis.

Figure 2 (a) The ACF image and the coordinate systems. (b) Prewitt mask operator. (c) 3 × 3 regional image (color figure available online).

The maximum direction of the numerical value increase of image f(x,y) direction is the gradient direction, and this increased direction can be written as α(x, y):

Another important physical quantity is the size of this vector, which is indicated as |∇f|, among which,

The gradient calculation used in this study is the improved Prewitt operators. The basic Prewitt mask utilizes two 3 × 3 masks, as shown in Figure 2 b, to be applied in the image with a convolution operation (Kindi and Shirinzadeh 2007).

Taking a 3 × 3 regional image as an example, as shown in Figure 2 c, z ₁ ∼ z ₉ are the gray-scale value of the pixel.

Then, the functional operation of the Prewitt mask can be written as:

Among which, g _x is the new gray-scale value which corresponds the original pixel z₅ to be differentiated in the horizontal direction through the Prewitt mask operation; g_y is the new gray-scale value which corresponds the original pixel z₅ to be differentiated vertically through the Prewitt mask operation.

While this study used the improved Prewitt mask operation, and was carried out with cross arithmetic processing of image for four kinds of 3 × 3 Prewitt mask, the operational formula can be displayed as follows:

g₁ ∼ g₄ are the new gray-scale value after each Prewitt mask operation.

After the operation, the final new pattern is taken to determine the fracture state of the particles. The image addition operation is f′(x,y) = f ₁(x,y) + f ₂(x,y); f′(x,y) is the new gray-scale value after the addition, f ₁(x,y) and f ₂(x,y) are the gray-scale value (Zhoua and Omarb 2009) of two images' corresponding pixel.

The image of conducting particles is processed by binarization before skeletonization, but the selection of its threshold affects the result of binarization treatment (Bilodeau and Bergevin 2005; Lin 1997).

Threshold k of two-value division divides all pixels of the whole image into two parts: C ₀ and C ₁, the variances of two aggregations are the following:

while

The probability density function is P _i , thus, the probability density of pixel's gray-scale value to be is i. The physical significance of the probability density summation ω₀ and ω₁ is the area of this aggregation in the histogram. μ₀ and μ₁ are the distance from the centroid of area of these aggregations such as C ₀ and C ₁ in the histogram to the ordinate axis. σ_W, σ_B, and σ _τ are within-class variance, between-class variance and total variance.

They are also the second moment of centroid of area of these aggregations and the ordinate axis. To find the optimal threshold k* of the discriminant criterion measurement is used in analysis to discuss and define the parameters:

The discriminant criterion should make σ _B maximum.

Obtaining parameter η is the easiest way to find threshold k.

Namely,

the optimized threshold k* is shown as

So, the optimal threshold of binary image can be obtained from the probability distribution. When the image is divided based on the optimized threshold k*, the second moment of the distance from the centroid of area of two aggregations, C ₀ and C ₁ to the centroid of area of original overall aggregation will be the maximum value.

For the image after binarization, the thinning processing method (Castleman 1998; Lim 1996) is like peeling an onion. Taking a 3 × 3 mask as an example, when it is on a 3 × 3 sub-image pixel; z ₅ is the center; the quantity of nonzero pixels in eight adjacent pixels is marked as N (z ₅).

For example, when z ₁ = 0, z ₂ = 0, z ₃ = 1, z ₃ = 1, z ₄ = 1, z ₆ = 1, z ₇ = 1, z ₈ = 0 and z ₉ = 1, then N(z ₅) = 5. Among which, if the pixel gray-scale value equals 0, in which the pixel is black; the pixel gray scale value equals 1, which means this pixel is white. For the eight adjacent pixels of z ₅, taking z ₁ as the start point, after returning to z ₁ along the clockwise direction, and calculating the times the pixel gray scale value changes from 0 to 1, it is marked as T(z ₅).

However, when N(z ₅) = 0 or 1, it means all the eight adjacent pixels of pixel z ₅ are white pixels, or only one pixel is a black pixel; so z ₅ may be an isolated point or the furthest peripheral terminal point. z ₅ will not be removed whether it is an isolated point or a terminal, namely when N(z ₅) = 1, z ₅ does not need to be changed to 0. When N(z ₅) = 0, there should no further processing. In other words, when 2 ≦ N(z ₅) ≦ 6 and z ₅ = 1, z ₅ needs further processing to determine whether z ₅ needs to be changed to 0.

Assuming 2 ≦ N(z5) ≦ 6 is validated, if T(z ₅) = 1 is also valid, when z ₅ = 1, z ₅ is likely to be changed to 0; to avoid the sinking of object thinning due to Z ₅ being changed to 0, two more conditions z ₂ · z ₆ · z ₈ = 0 and z ₂.z ₄. z ₆ = 0 are made, so the sinking towards north and east when the black pixel z ₅ is removed can be prevented. Based on the above, determining whether z ₅ should be changed to a black pixel with gray-scale value of 0 is based on the four equations as shown in the following:

Then, when the above four equations are validated simultaneously, and z ₅ = 1, z ₅ will be changed to 0.

However, when thinning objects, the symmetry of thinning still needs to be considered. (C) and (D) of Equation (20) tend to thinning processing in the southeast. So, to balance the thinned skeleton, this study carried out west and north thinning alternately, and the four differential equations are as follows:

When carrying out practical objects thinning, this study started the thinning processing at the furthest periphery of the object, and then carried out thinning in the other two directions according to Equation (21). It alternated the two equations for computation, and carried out continuous thinning operation at the periphery of the object, until the skeleton of the object has 1 pixel width left to complete the skeletonization.

3. EXPERIMENTAL RESULTS

Figure 3 shows the relations of conducting particle shape, conditions, and its deformation. The deformation degree and conducting conditions can be evaluated by the particle shape. As shown in Figure 4 a, it is an anisotropic conductive film inspection image with only the protruding shapes of the conducting particles. All conducting particles are distributed in the hexagonal bump block. Therefore, when selecting a block template image as a target template, only one needs to be selected to define the average gray level and the outline of the target, as shown in Figure 4 b. After the image pretreatment, the block in which all conducting particles are distributed, can be obtained, as shown in Figure 4 c. Various conducting particles can be obtained from it, as shown in Figure 4 d.

Figure 3 Relations of conducting particle shape, conditions and its deformation (color figure available online).

Figure 4 (a) ACF inspection image. (b) Target template. (c) Comparison result of block target template. (d) Comparison result of conducting particles (color figure available online).

However, when the targets to be inspected are distributed in diversified block images, as shown in Figure 5 a, the block templates need to be increased, as shown in Figure 5 b, to carry out synchronous multiple comparison and identification for the image to be inspected. The comparison results are shown in Figure 5 c. As shown, the diversified bump blocks in which all conducting particles are distributed are discriminated.

Figure 5 (a) ACF inspection image of the conducting particles distributed in diversified blocks. (b) Settings for complex template. (c) Result display of synchronous comparison of complex template (color figure available online).

On anisotropic conductive film image pretreatment, there is a problem in image quality and noise distribution, so gray scale, close operation, binarization, improved Prewitt mask, and local scope limitation are used for alternate processing, to increase the contrast between the conducting particle image and the background. In the Figure 6 a pretreatment series (1), the pretreatment process, from left to right, is, original image, binarization with discriminant criterion, and close operation. The primary conducting particle image is separated from the background after the processing, binarization image is formed, and the local range restriction is utilized for filtering the rest of the noise, so a preliminary analysis and judgment of conducting particle image can be completed. The pretreatment series (2) is shown as Figure 6 b. The pretreatment process, from left to right, is the original image, gray scale and the improved Prewitt mask operation. This is the necessary procedure before skeletonizing the conducting particle image. Figure 6 c is the binarization image obtained after each conducting particle image with the improved Prewitt mask and the binarization of discriminant criterion.

Figure 6 (a) Image pretreatment series (1). (b) Image pretreatment series (2). (c) Pretreatment of conducting particle image (color figure available online).

When a binarization image of a conducting particle image is obtained after the pretreatment process, the skeletonization processing is carried out, as shown in Figure 7. The treatment scheme, from left to right, is, original conducting particle image, binarization, improved Prewitt mask operation, and skeletonization processing.

Figure 7 Skeletonization treatment scheme (color figure available online).

However, when the system analyzes the fracture characteristics of conducting particles of anisotropic conductive film image, it sets the bump block template image at first, as shown in Figure 8 a, and sets the selection of conducting particle image template, as shown in Figure 8 b. Finally, it executes the result of the comparison operation, as shown in Figure 8 c. As shown, all bump blocks are compared, and 0 ∼ 5 are marked, which represent the position of bump blocks from Target 0 to Target 5. The fracture edge of the conducting particles is marked by the line, and the quantity and amount information of each kind of conducting particles distributed in various targets can be obtained. Figure 9 is the image processing result of the distribution of each conducting particle. The coordinate position of each conducting particle distributed in each target, as well as the fracture edge length, can be obtained to estimate the fracture state of the conducting particles. According to the above data, the repeatability of this system can be expected to lie within 10% error probability, which is a significant improvement. It can be verified in an automatic LCD assembly production line.

Figure 8 (a) Set Bump block template image. (b) Set selection of conducting particle image template. (c) Comparison result of fracture characteristics of ACF conducting particles (color figure available online).

Figure 9 Result of ACF conducting particles distribution (color figure available online).

4. CONCLUSION

The primary function of the anisotropic conductive film is to provide the conductance of two joiners vertically, while there is insulation effect in the horizontal direction. It is mainly composed of conducting particles and binder, and makes the conducting particles of anisotropic conductive film fractured and deformed through the control of temperature, pressure, and joint displacement, so a conduction circuit is formed.

However, due to differences in machines, and different process conditions and conducting resin materials, various deviations are caused after press fitting, such as the quantity of conducting particles, fracture state, and the joint displacement. The deviation of the anisotropic conductive film joint position affects the conduction between the whole LCD panel and control circuit, resulting in seal OK or NG. Therefore, it is very important to develop the inspection technology for conducting particles.

Since the assembly process of liquid crystal displays is very complicated, much support of optical detection devices and image processing systems is needed to improve the quality of products and yield rate. This study utilizes the assistance of image processing technology to decrease the defection rate of products, and can be verified in an automatic production line.

ACKNOWLEDGMENTS

This work was sponsored by the Taiwan National Science Council under grant number NSC 99-2221-E-035-081.