CNN-Cap: Effective Convolutional Neural Network Based Capacitance Models for Full-Chip
Parasitic Extraction

For full-chip capacitance extraction, directly using the 3-D field solver is infeasible due to its excessive cost of memory and CPU time. To obtain good tradeoff between accuracy and efficiency, the 2.5-D extraction method with pattern matching technique is widely used. The pattern matching based extraction methodology include three major modules: 1) pattern generation, 2) capacitance model building, and 3) layout capacitance extraction [2, 9, 15]. It is illustrated as Fig. 1. For a given process technology, the work of 1) pattern generation and 2) capacitance model building are carried out just once.

The 2.5-D extraction method refers to the method considers 3-D geometric effects on capacitance with 2-D interconnect patterns through sweeping 3-D geometry of interconnects in two perpendicular directions [16, 17]. It is adopted by the OpenROAD project for parasitic extraction [18]. During the layout capacitance extraction, extraction windows are generated along the interconnect line (called master net or master conductor) whose capacitances are of concern. Each window includes a segment of the master net and its neighbor conductors (called environmental conductors). The techniques of 2.5-D extraction and pattern matching are employed to calculate the capacitances among the conductors in the window. Take the structure shown in Fig. 2 as an example, where a wire with name $m2$ crosses over a wire named $m1$. Along direction A, a 2-D cross-section view is shown in the middle of Fig. 2. Along direction B, the other 2-D cross section is shown to the right. If the capacitances in the two 2-D cross-section views are known, we can approximately compute the capacitance between $m1$ and $m2$ with the 3-D effects taken into consideration. Suppose the 2-D capacitance between $m1$ and $m2$ in the view along A is

where $C_{1f1}$ and $C_{1f2}$ are two fringe capacitances, and $C_{1o}$ is the overlapping capacitance. Similarly,

for the view along B. Then,

where $w_{1}$ and $w_{2}$ are widths of wires $m1$ and $m2$, respectively.

With this method, one can only consider the capacitance models for the 2-D cross-section structures. These structures are regarded as the instances of pre-defined 2-D patterns. Two kinds of typical structure patterns are shown in Fig. 3. Pattern-A shown in Fig. 3 is a “sandwich” structure including two big-plane conductors and three parallel wires in between. The red block denotes the master conductor, and the whole structure is usually left-right symmetrical. Pattern-B’s structure is more general than A, where the conductors on the top and down metal layers are not required to be a big plane across the extraction window. On the layer where the master conductor lies, there are two conductors on the left and right side of the master respectively. At the very bottom, there is an extra big-plane conductor representing the substrate ground. Although not shown in Fig. 3, the multi-dielectric environment is considered in these structure.

Usually in a window, only the conductors located on the nearest metal layers above and blow the master conductor are considered. Due to the proximity effect of electrostatic field, this brings little error to the capacitances of the master. Therefore, most patterns are defined by three metal layers (containing master conductor and its neighbour conductors) in a given process technology, and the numbers of conductors on each layer. A capacitance model for a pattern is built through computing the capacitances of a lot of instance structures with field solver and then building capacitance models with look-up tables or curve fitting techniques.

The neural network with multiple hidden layer of neurons is often refers to as deep neural network (DNN). Let $\boldsymbol{x}\in\mathbb{R}^{n}$ be the input data and $\boldsymbol{y}\in\mathbb{R}^{m}$ be the output data, then, the DNN can be viewed as a function $f(\boldsymbol{x};\boldsymbol{\theta}):\mathbb{R}^{n}\rightarrow\mathbb{R}^{m}$ with parameters $\boldsymbol{\theta}$. The form of the function $f$ determines the type of DNN. The training of a DNN and the prediction with a trained DNN are illustrated in Fig. 4. During the training, a large amount of input data along with the corresponding outputs (called labels) are fed to the network. The network parameters $\theta$ are optimized to minimize the difference between $f(\boldsymbol{x};\boldsymbol{\theta})$ and the corresponding labels. When the difference is sufficiently small, the network is trained to become a good regressor, which can be used to do prediction (see Fig. 4(b)).

The MLP neural network is a simple yet popular neural network, as shown in Fig. 5 where only one hidden layer is drawn. Suppose there are $n_{l}$ hidden layers. Let $\boldsymbol{h}^{(i)}$ be the variables residing at $i$-th hidden layer’s neurons. Those on the input layer and output layer can be denoted by $\boldsymbol{h}^{(0)}$ and $\boldsymbol{h}^{(n_{l}+1)}$, respectively. The input data elements are assigned to each neuron in the input layer and feed-forward to the next layer iteratively until they reach the output layer. For the $i$-th layer, it means $\boldsymbol{h}^{(i)}=g_{i}(\boldsymbol{h}^{(i-1)})$ where $g_{i}$ is a function to represent the feed-forward process. In general, $g_{i}$ is a compound function of a nonlinear activation function and a linear function including weight parameters.

The MLP network has been studied in capacitance extraction of several 3-D structures [13]. The idea is straightforward and can be applied to build the capacitance model of the 2-D patterns. For Pattern-A, there are just three width/spacing parameters for describing the structure: $w_{1},w_{2}$ and $s$ (see Fig. 3). For the example of Pattern-B (see 3), the parameters are widths $w_{1},w_{2},w_{3},w_{4},w_{5}$, and location coordinates $x_{2},x_{3},x_{4},x_{5}$. This makes the regression model well defined; the input $\boldsymbol{x}$ is just a vector including these geometry parameters, and the output $\boldsymbol{y}$ is a vector including the total capacitance and coupling capacitances. With a lot of sample structures for a pattern and corresponding capacitances results from field solver, the MLP based model can be trained.

Besides training approach and model architecture, different loss function also affects the difficulty of optimization process and the performance of trained model. Usually the mean square error (MSE) is employed as the loss function for minimization:

where $N$ is the number of training examples, $\boldsymbol{x}^{(i)}$ indicates the $i$-th example, and $\boldsymbol{y}^{(i)}$ is the corresponding label.

In all existing methods, a pattern in the 2.5-D extraction flow is determined by a combination of metal layers and the numbers of conductors on these layers. And, the geometric parameters characterizing a pattern increases with the number of conductors in the pattern. For a pattern with a large quantity of parameters, building an accurate model with either traditional approaches or the MLP based approach becomes difficult. To overcome this issue, we view the 2-D cross-section structure as a kind of image, and try to characterize capacitance related information within it with the convolutional neural network which performs very well in image processing applications. So, in this way, we can allow a pattern including a variable number of conductors and thus largely reduce the number of patterns.

This considered pattern is illustrated by Fig. 6. Its structure is more general than that of Pattern-B. The master conductor is still at the center of the middle layer. Although the conductors can lie on more than three metal layers, we assume they are just on three metal layers (with index $i$, $j$ and $k$) in the rest of this paper. The proposed method can be easily extended to the patterns with more than three metal layers. The structure in Fig. 6 is referred to as Pattern-C.

Inspired by the idea of encoding the layout of interconnect wires with density-based features in the design for manufacture (DFM) research [19], we propose to evenly divide the width of extraction window into $L$ grid cells. Thus, the conductor placement of a metal layer can be depicted by an $L$-dimensional vector. The value of vector element is the density, i.e. the ratio of conductor occupying the grid cell. See the example in Fig. 7, where the horizontal placement of three conductors (labeled 1, 2 and 3), the grids, and the corresponding density map are shown. For a Pattern-C structure, the conductor placement can be described with three $L$-dimensional vectors. Notice this representation is not ambiguous if we guarantee that the grid size is less than the minimum spacing between wires on the metal layer.

This grid-based representation cannot identify the master conductor. So, we must encode more information regarding the pattern into it. Suppose the structure includes $n_{c}$ conductors. Our problem is to extract one total capacitance, and $n_{c}-1$ coupling capacitances. Consider the density-based representation for one metal layer: $\boldsymbol{d}\in\mathbb{R}^{L}$. We modify it to obtain an $L$-dimensional vector $\boldsymbol{x}$ for a sub-problem of calculating total capacitance. The scheme of the modification is that, if the master conductor covers the $i$-th grid, we have $x_{i}=d_{i}+1$. The obtained feature vector $\boldsymbol{x}$ is shown as the third row in Fig. 7. For calculating the coupling capacitance between the master and an environmental conductor, a unique feature vector is also generated by modifying $\boldsymbol{d}$. Besides adding 1 to the elements corresponding the cells overlapped with the master, another modification is applied to identifying the environmental conductor. If the environmental conductor covers the $i$-th grid, we make that $x_{i}=-d_{i}$. Now, for calculating different coupling capacitance there is a different feature vector for data representation. In Fig. 7, the last two rows show the feature vectors for the sub-problems of calculating coupling capacitances $C_{12}$ and $C_{13}$.

Because the values of total capacitance and coupling capacitance can differ for several orders of magnitude, the accuracy demand for them is usually different. In this work, we distinguish the problems of calculating total capacitance and of coupling capacitances, and trains two models for them respectively for a given Pattern-C. This ensures the overall accuracy of capacitance extraction. With this idea and the proposed grid-based data representation, our method for building the capacitance models can be depicted as Fig. 8. Notice that a sample structure is converted to $n_{c}$ data inputted to the two models in the training stage.

And, in the prediction stage, the trained model Model-C_C for coupling capacitance will be evaluated for $n_{c}-1$ times to output the $n_{c}-1$ coupling capacitances.

Because the CNN is able to capture the spatial information in the data, it is expected to perform better than MLP neural network for the problem of capacitance calculation. So, we develop the CNN base models (called CNN-Cap) for predicting the capacitances of Pattern-C structures. There are two models with same architecture used for total capacitance and coupling capacitance, respectively.

The architecture of CNN-Cap is shown in Fig. 9, which is derived from the famous ResNet architecture [20]. In our problem, the input data (pattern structure with extraction information) is vectors, instead of the matrices in image related problem. So, we just use 1-D convolutional layer (shown as colored block in Fig. 9) in CNN-Cap. Notice that the number of metal layers in the pattern is like the concept of “channel” of image data. A batch normalization and a ReLU layer are inserted after every convolutional layer. The input data will reach the last convolutional layer through stacked convolutional layers.

Then, it will be pooled into a fixed-length vector by an average pooling layer. Finally, the fixed-length vector will be fed to the fully-connected layer with an output dimension of one to predict capacitance. In Fig. 9, the convolutional layer captioned “1x3 conv, 64” means it has 64 channels and a kernel size of 3. “/2” means to halve the input length (by a 1-D pooling layer or a 1-D convolutional layer with a stride of 2). The convolutional layers with the same color mean they have the same input length. Whenever a 1-D convolutional layer halves the input length, the input channel will be doubled. The key point of CNN-Cap is the shortcut connection, which takes a shallower vector $\boldsymbol{h}_{1}$ and a deeper vector $\boldsymbol{h}_{2}$ as input and takes $\mathcal{F}(\boldsymbol{h}_{1})+\boldsymbol{h}_{2}$ as output. The function $\mathcal{F}$ is an identity mapping in most cases (solid line). The dotted shortcut takes a 1-D 1x1 convolutional layer (equivalent to linear projection) as the mapping function when the shape of two input vectors is mismatched. Without the shortcut connection, a deeper DNN model will have both higher testing error and training error. The higher training error means that it is difficult to train a deep plain model. The shortcut connection can preserve the low-level features all the time, which makes training easier. Thanks to the advanced modules, i.e., 1-D convolutional layer and shortcut connect, our CNN-Cap has better learning ability than the MLP neural networks. This will be revealed with experimental results.

We have tested three architectures derived from ResNet-18, ResNet-34, and ResNet-50. We found out that derived from ResNet-34 is consistently better than that from ResNet-18, and performs similarly to that from ResNet-50. So, the ResNet-34 derived architecture is chosen in this work.

For training CNN-Cap, the stochastic gradient descent (SGD) optimizer and the Adam optimizer [21] are considered. Our experimental results show the Adam optimizer makes convergence easier to reach and the trained model perform better. So, it is employed in this work.

The approach of grid search is used to find the appropriate learning rate and batch size, which affect performance of the obtained CNN-Cap model. The value range of learning rate is from $10^{-5}$ to $10^{-2}$, and the batch size is enumerated in $\{16,32,64,128\}$.

For the cost function, in addition to the MSE loss function (4), we have tried the following loss function.

where $N$ is the number of training data, $\boldsymbol{x}^{(i)}$ indicates the $i$-th input data, and $\boldsymbol{y}^{(i)}$ is the corresponding label. The division of (5) stands for an element-wise operation. This MSRE loss function includes the relative error, so that the optimization could possibly attain a better accuracy. However, it may bring difficulty to the convergence of training process. We have done extensive experiments, and found out that the loss function MSRE is better than MSE for the task of training the Model-C_C for coupling capacitance.

In this work, a data is a 2-D structure of cross-section view with the proposed data representation, and the label includes the capacitances computed with a field solver.

For a given process technology and a specified triple of ($i$, $j$, $k$) of metal layer indices, we can generate random sample structures of Pattern-C. We ensure that the data complies with the rules of minimum width and minimum spacing of the process technology. As the structure is a cross-section view along the interconnect line, we let the width of master conductor be a smaller value with larger probability and not exceed 10 times of the minimum width. Similarly, the random data can be generated for the conventional patterns, like Pattern-A and Pattern-B. Notice we can also build the CNN-Cap model for a conventional pattern.

The whole width of the extraction window is determined with a simulation experiment to detect how far if an environmental conductor is away from the master conductor the coupling capacitance between them decreases to 1% of the total capacitance.

Fig. 10 shows the geometries of the conductors in two data generated with our approach. Fig. 10 stands for a data for Pattern-B and Fig. 10 is for Pattern-C.

The existing approach with MLP neural network cannot handle Pattern-C, because the structure involves a variable number of conductors. Instead, we can combine the MLP with the grid-based data representation. It means we connect the three vectors to a form a long vector and then input it to the MLP network. However, this hybrid model cannot have high accuracy, because the spatial information in the grid-based representation is lost. The comparison experiments in Section IV.B validate this.

For Pattern-B structures, the MLP-based model presented in Section II.B can be used as the baseline. We call it MLP-Cap. Similar tuning is also applied to MLP-Cap, with a dataset of Pattern-B. For the architecture, we tried different numbers of hidden layers and different numbers of neurons respectively. After some heuristic trials, we finally use an architecture with three hidden layers, and 256, 256 and 512 neurons are set in the three layers respectively. Three nonlinear activation functions: ReLU, Sigmoid and Tanh are tested. The results show that Tanh function consistently surpass the other two. The grid search is used to find the appropriate learning rate and batch size. Finally, the batch size is set to $32$, and the learning rate is set to $10^{-5}$. Besides, the loss functions fo MSE (4) and MSRE (5) are tested with MLP-Cap. The results show that with the MSE loss function makes MLP-Cap perform better.

In this experiment, the pattern is the same as that in Fig. 3(b) which includes 1, 3 and 1 conductors on three metal layers respectively. And, the structure can be described with 9 geometric parameters, which are the input to MLP-Cap.

The results for predicting total capacitance and coupling capacitance, for eight datasets from the both technologies, are shown in Table II and Table III, respectively. In the tables, every two rows correspond to a data set generated for a pattern. $Err_{avg}$ means the average of the relative error’s absolute value. $Err_{max}$ means the maximum of the relative error’s absolute value. Ratio(Err$>$5%) in Table II means the ratio of the number of total capacitances with error larger than 5% to the number of all total capacitances predicted. Ratio(Err$>$10%) in Table III means the ratio of the number of coupling capacitances with error larger than 10% to the number of all coupling capacitances predicted. Here, we regard relative error of 5% as the accuracy criterion for total capacitance, and the relative error of 10% as the accuracy criterion for coupling capacitance.

The experimental results show that CNN-Cap always performs better than MLP-Cap. In more details, both CNN-Cap and MLP-Cap can predict total capacitance with less than $1\%$ relative error on average, while the maximum relative error of CNN-Cap is not larger than 1.3%. However, the maximum relative error of MLP-Cap is often larger than $5\%$. As for predicting coupling capacitance, the maximum relative error of MLP-Cap is always larger than 10% and can be as large as 114%. On the contrary, only a very few of coupling capacitances (at most 0.1% of all coupling capacitances) computed with CNN-Cap have relative error larger than $10\%$.

Fig. 11 shows the coupling capacitances calculated with MLP-Cap and CNN-Caps respectively, along the relative error of each capacitance. From the figure we see that CNN-Cap can accurately predict most coupling capacitance, and the few examples of inaccurate prediction usually corresponds to a small coupling capacitance. This means reasonable and acceptable accuracy. On the contrary, the MLP-Cap is unsatisfactory in many cases, and its maximum error is very large as shown in Fig. 11.

More dataset for different layer combinations have been generated and tested. The results show that for all the tested datasets, the average relative error of CNN-Cap on total capacitance and coupling capacitance are just 0.22% and 1.36%, respectively. This experiment demonstrates the good accuracy of CNN-Cap for Pattern-B structures.

Another capacitance modeling approach which may have good accuracy is based on look-up table and the bilinear interpolation. For the considered Pattern-B structures, if each parameter takes 20 sample values the total numbers stored in the look-up table would be $20^{9}=5.12\times 10^{11}$, which leads to unacceptable memory cost in reality. This demonstrates the advantage of capacitance modeling technique based on neural networks.

For generating the sample structures of Pattern-C, we assume the total number of conductors on the up and down metal layers ranges from 6 to 8. Such a pattern with a variable number of conductors cannot be handled by a single MLP-Cap. So, we consider the hybrid method discussed at the end of Section III as the baseline. It is called Grid+MLP model. It utilizes the grid-based data representation and shares the same architecture and hyper-parameters as MLP-Cap. And, it is trained with the same approach for MLP-Cap.

The results of CNN-Cap and Grid+MLP model for four datasets of Pattern-C are listed in Table IV and Table V. The results show that CNN-Cap performs much better than Grid+MLP. The maximum relative error of CNN-Cap on total capacitance is not larger than 1.2%. As for coupling capacitance, the performance of the Grid+MLP is much worse and unacceptable, with the maximum error as large as 542%. On the contrary, the relative error derived from the CNN-Cap is less than $10\%$ for more than 99.5% predicted capacitances.

Fig. 12 shows the calculated coupling capacitance (by CNN-Cap) versus relative error for Pattern-C structures. It reveals again that CNN-Cap has very good accuracy on coupling capacitance for most structures. For example, for the test data corresponding to layer combination (2, 3, 6) in the 55nm technology, only 0.3% of all coupling capacitances has error larger than 10%.

More dataset for different layer combinations have been generated and tested. The results show that, for all the tested datasets the average relative error of CNN-Cap on total capacitance and coupling capacitance are just 0.18% and 1.38%, respectively. This experiment demonstrates the good accuracy of CNN-Cap for Pattern-C structures.

An additional experiment is carried out to evaluate the effect of training set’s size on CNN-Cap’s accuracy. We change the ratio of the training subset from 90% to 80%, 70%, down to 10%, and then rerun the training process for a dataset of Pattern-C, respectively. For each resulted CNN-Cap model, we examine it with the testing subset. The average relative error on coupling capacitance and the ratio of the coupling capacitances with error larger than 10% are plotted in Fig. 13.

From it we see that, when the ratio of training subset is 50% or larger (meaning with 25000 data or more) the trained CNN-Cap model always has fairly good accuracy. Similar results are observed for the total capacitance. Therefore, a training set with 25000 data through 45000 data is usually sufficient for building an effective CNN-Cap model.

We compare the prediction time of CNN-Cap with Raphael, i.e. the 2-D field solver rc2 within Raphael. For a dataset including 50000 Pattern-C structures, we extract the capacitances of them with CNN-Cap and Raphael respectively. Their average runtimes per structure are listed in Table VI. From the table we see that CNN-Cap runs 4693X faster than Raphael. Even though we can run multiple processes of Raphael on the machine, say 32 processes, calculating capacitance with CNN-Cap can still be 147X faster than running the 2-D field solver.

As for the model size, a CNN-Cap model has 14473418 parameters occupying 13.8 MB storage, which is much smaller than the look-up table based model. The training time for a CNN-Cap model is about 1.3 hours. Considering various layer combinations for a given process technology with 10 metal layers (like that in FreePDK15), we see that building all CNN-Cap models can be completed within a week on a GPU server with 8 Nvidia RTX2080Ti GPUs. And, the CNN-Cap models deliver much better accuracy than the existing methods for pattern capacitance modeling.