A Novel Nonlinear Non-parametric Correlation Measurement With A Case Study on Surface Roughness in Finish Turning

Ming Luo Srinivasan Radhakrishnan Sagar Kamarthi Northeastern University, 360 Huntington Avenue, Boston, Massachusetts, United States

Abstract

Estimating the correlation coefficient has been a daunting work with the increasing complexity of dataset’s pattern. One of the problems in manufacturing applications consists of the estimation of a critical process variable during a machining operation from directly measurable process variables. For example, the prediction of surface roughness of a workpiece during finish turning processes. In this paper, we did exhaustive study on the existing popular correlation coefficients: Pearson correlation coefficient, Spearman’s rank correlation coefficient, Kendall’s Tau correlation coefficient, Fechner correlation coefficient, and Nonlinear correlation coefficient. However, no one of them can capture all the nonlinear and linear correlations. So, we represent a universal non-linear non-parametric correlation measurement, $g$ -correlation coefficient. Unlike other correlation measurements, $g$ -correlation doesn’t require assumptions and pick the dominating patterns of the dataset after examining all the major patterns no matter it is linear or nonlinear. Results of testing on both linearly correlated and non-linearly correlated dataset and comparison with the introduced correlation coefficients in literature show that $g$ -correlation is robust on all the linearly correlated dataset and outperforms for some non-linearly correlated dataset. Results of the application of different correlation concepts to surface roughness assessment show that $g$ -correlation has a central role among all standard concepts of correlation.

keywords:

Association, predictability , Fechner correlation coefficient , pattern recognition , surface roughness

1 INTRODUCTION

A classical problem in statistics for ordered pairs of measurement $(x_{1},y_{1}),(x_{2},y_{2}),\\ \dots,(x_{n},y_{n})$ is that is it possible in general from knowing one of the values in an arbitrary pair $\,(x,~{}y)\,$ of measurements to draw conclusions about the other value in this pair.

When speaking about correlation the statistics literature mainly aims at finding a certain functional relationship (such as a straight line in linear regression) or a monotonic relationship between two numerical variables. In General, correlation analysis is a means of measuring the strength or ‘closeness’ of the relationship between two variables [2].

In section 2 we try to provide general definitions and clarifications of the terms correlation and predictability for pairs of random variables. The most important methods for measuring a correlation between two random variables are also presented and discussed in section 2. Two concepts for nonparametric correlation analysis and prediction $--$ a seldomly used one and the recently introduced $g$ -correlation $--$ which can detect correlations which are neither functional nor monotonic relationships are described, derived and analyzed in sections 3 and 4. In section 5 it is shown how the $g$ -correlation concept can be generalized for more than 2 variables. In section 6 the $g$ -correlation is compared with among all the correlation measures introduced in the paper with linearly and nonlinearly correlated dataset. In section 7 the $g$ -correlation is compared with other important correlation measures for a problem of surface roughness prediction in finish turning and the results are summarized in section 8.

2 THE CONCEPT OF CORRELATION BETWEEN TWO VARIABLES

When prediction is a concern in correlation analysis, statistician make the following distinction [9]: Correlation is a measure of degree of dependence between two dependent random variables.

Definition 1 A dependent random variable $Y$ is uncorrelated with respect to an independent random variable $X$ if the range and/or frequency of the possible values for $Y$ is constant for varying values of $X$ . In mathematical terms, this means that, for any two unique values $x_{i},~{}x_{j}$ in $X$ , $\,F_{Y|X}(y|x_{i}):=P(Y\leq y|X=x_{i})\,$ and $\,F_{Y|X}(y|x_{j}):=P(Y\leq y|X=x_{j})\,$ are equivalent for all $y$ in $Y$ .

Correlation therefore means that the values of the dependent variable $Y$ do not always behave completely randomly. In fact, their distribution is influenced by and can be predicted from the corresponding value of $X$ . It is shown with the following Lemma 2 that under some technical assumptions the correlation defined by Definition 1 is equivalent to the well-known concept of statistical dependence of two random variables – one random variable has a certain impact on the other random variable and vice versa.

Lemma 2 Let $X$ and $Y$ be two random variables which are either a) both discrete or b) both continuous with a joint density function $f$ . Under this assumption, $X$ and $Y$ are uncorrelated according to Definition 1 if and only if they are statistically independent.

Proof.

To prove it, we will show that the statement that X and Y are uncorrelated is both necessary and sufficient for statement that X and Y are statistically independent in both case a) and case b).

Case a: The discrete case means that $X$ and $Y$ take finitely or countably many values $\,x_{1},~{}x_{2},~{}\dots\,~{}x_{j},~{}\dots\,~{}x_{n}$ and $\,y_{1},~{}y_{2},~{}\dots,~{}y_{i},~{}\dots\,~{}y_{n}$ , respectively, which are listed in increasing order. In this case statistical independence of $X$ and $Y$ is defined as

{\rm P_{Y,X}}\{Y=y_{i},~{}X=x_{j}\}={\rm P_{Y}}\{Y=y_{i}\}{\rm P_{X}}\{X=x_{j}\}

(1)

for all indices $\,i,j$ .

Necessary: If $X$ and $Y$ are independent, then the conditional probability distribution function of $Y$ given $X=x,$ $F_{Y|X}(y|x),$ is defined as

F_{Y|X}(y|x)={\rm P}\{Y\leq y|X=x\}=\frac{{\rm P}\{Y\leq y,~{}X=x\}}{{\rm P}\{X=x\}}

(2)

for all $x\in\{\,x_{1},~{}x_{2},~{}\dots\,~{}x_{j},~{}\dots\,~{}x_{n}\}$ such that ${\rm P}\{X=x\}>0$ . This can be further written as

F_{Y|X}(y|x)=\frac{\sum_{y_{k}\leq y}{\rm P}\{Y=y_{k},~{}X=x\}}{{\rm P}\{X=x\}},y_{k}\in\{\,y_{1},~{}y_{2},~{}\dots,~{}y_{i},~{}\dots\,~{}y_{n}\}

(3)

and using equation (1) we further get

F_{Y|X}(y|x)=\frac{\sum_{y_{k}\leq y}{\rm P}\{Y=y_{k}\}{\rm P}\{X=x\}}{{\rm P}\{X=x\}}=\sum_{y_{k}\leq y}{\rm P}\{Y=y_{k}\}=P\{Y<y\}.

(4)

This means that $\,F_{Y|X}(y|x)\,$ is always the same function independent of the value $\,x$ , i.e. $X$ and $Y$ are uncorrelated.

Sufficient: For the opposite direction,

(a)

we first consider the case where $X$ takes only a single value $x_{1}$ with probability 1. Then, clearly,

${\rm P}\{Y=y_{i},~{}X=x_{1}\}={\rm P}\{Y=y_{i}\}={\rm P}\{Y=y_{i}\}{\rm P}\{X=x_{1}\}$ (5)

for every index $i$ range from 1 to $n$ , which is just the independence of $X$ and $Y$ .
(b)

Secondly, assume that

$F_{Y|X}(y|x_{k})={\rm P}\{Y\leq y|X=x_{k}\}={\rm P}\{Y\leq y|X=x_{j}\}=F_{Y|X}(y|x_{j})$ (6)

for every real value $y$ and every pair of different indices $\,k,j\in\{1,2,3,\dots,n\}$ .

Equation (6) can be reformulated as

		$\displaystyle\frac{{\rm P}\{Y\leq y,~{}X=x_{k}\}}{{\rm P}\{X=x_{k}\}}=\frac{{\rm P}\{Y\leq y,~{}X=x_{j}\}}{{\rm P}\{X=x_{j}\}}$
	$\displaystyle\Longleftrightarrow\phantom{aaa}$	$\displaystyle\frac{\sum_{y_{l}\leq y}{\rm P}\{Y=y_{l},~{}X=x_{k}\}}{{\rm P}\{X=x_{k}\}}=\frac{\sum_{y_{l}\leq y}{\rm P}\{Y=y_{l},~{}X=x_{j}\}}{{\rm P}\{X=x_{j}\}}.$		(7)

Substituting $y$ by the values $y_{i}$ and $y_{(i-1)}$ for an arbitrary index $i$ leads to

\frac{\sum_{y_{l}\leq y_{(i-1)}}{\rm P}\{Y=y_{l},~{}X=x_{k}\}}{{\rm P}\{X=x_{k}\}}=\frac{\sum_{y_{l}\leq y_{(i-1)}}{\rm P}\{Y=y_{l},~{}X=x_{j}\}}{{\rm P}\{X=x_{j}\}}

(8)

and

\frac{\sum_{y_{l}\leq y_{i}}{\rm P}\{Y=y_{l},~{}X=x_{k}\}}{{\rm P}\{X=x_{k}\}}=\frac{\sum_{y_{l}\leq y_{i}}{\rm P}\{Y=y_{l},~{}X=x_{j}\}}{{\rm P}\{X=x_{j}\}}.

(9)

Subtracting equation (8) from (9) gives

\frac{{\rm P}\{Y=y_{i},~{}X=x_{k}\}}{{\rm P}(\{X=x_{k}\}}=\frac{{\rm P}\{Y=y_{i},~{}X=x_{j}\}}{{\rm P}\{X=x_{j}\}}

(10)

, which is equivalent to

{\rm P}\{Y=y_{i},~{}X=x_{k}\}=\frac{{\rm P}\{Y=y_{i},~{}X=x_{j}\}}{{\rm P}\{X=x_{j}\}}\cdot{\rm P}(\{X=x_{k}\}.

(11)

In addition, for each index $i$ and consequently

$\displaystyle{\rm P}\{Y=y_{i}\}$	$\displaystyle=\sum_{k=1}^{n}{\rm P}\{Y=y_{i},~{}X=x_{k}\}$	(12)
	$\displaystyle=\sum_{k=1}^{n}\frac{{\rm P}\{Y=y_{i},~{}X=x_{j}\}}{{\rm P}\{X=x_{j}\}}\cdot{\rm P}(\{X=x_{k}\}$
	$\displaystyle=\frac{{\rm P}\{Y=y_{i},~{}X=x_{j}\}}{{\rm P}\{X=x_{j}\}}\sum_{k=1}^{n}{\rm P}\{X=x_{k}\}$
	$\displaystyle=\frac{{\rm P}\{Y=y_{i},~{}X=x_{j}\}}{{\rm P}\{X=x_{j}\}}.$

This implies that

{\rm P}\{Y=y_{i},~{}X=x_{j}\}={\rm P}\{Y=y_{i}\}{\rm P}\{X=x_{j}\}

(13)

for all $i$ and $j$ , which means that the random variables $X$ and $Y$ are independent.

Case b: In the continuous case, the statistical independence of $X$ and $Y$ is equivalent to the relation

f_{YX}(y,x)=f_{Y}(y)f_{X}(x)\quad\mbox{almost everywhere,}

(14)

where $f_{Y}$ and $f_{X}$ are the marginal probability density functions of $Y$ and $X$ respectively [7].

1.

Necessary: The conditional probability density function of $Y$ given $X=x$ is defined by

$f_{Y|X}(y|x):=\frac{f_{YX}(y,~{}x)}{f_{X}(x)}$ (15)

for every value $x$ with $\,f_{X}(x)>0\,$ [4]. If $X$ and $Y$ are independent, then it follows from equation (14) that

$f_{Y|X}(y|x)=f_{Y}(y)\quad\mbox{almost everywhere}$ (16)

for every possible value $x$ with $\,f_{X}(x)>0$ . Thus the conditional cumulative distribution function of $Y$ given $X=x$ is

$\,F_{Y|X}(y|x):=P(Y\leq y|X=x)=\int_{-\infty}^{y}f_{Y|X}(u|x)\,\mbox{d}\mu(u)\,$

are all equal if $x$ is varied. Consequently, by Definition 1, $X$ and $Y$ are uncorrelated.

Sufficient: Conversly, if we assume that

F_{Y|X}(y|x_{i})=F_{Y|X}(y|x_{j})

(17)

for every pair of values $\,x_{i}\neq x_{j}\,$ for which $\,F_{Y|X}\,$ can be defined. We get

f_{Y|X}(y|x_{i})=F_{Y|X}^{\prime}(y|x_{i})=F_{Y|X}^{\prime}(y|x_{j})=f_{Y|X}(y|x_{j})\quad\mbox{almost everywhere}.

(18)

If we fix $x_{i}$ arbitrarily, then

	$\displaystyle\frac{f_{YX}(y,~{}x_{i})}{f_{X}(x_{i})}$	$\displaystyle=\frac{f_{YX}(y,~{}x_{j})}{f_{X}(x_{j})}\quad\mbox{almost everywhere}$		(19)
	$\displaystyle\Longleftrightarrow f_{YX}(y,~{}x_{j})$	$\displaystyle=\frac{f_{YX}(y,~{}x_{i})}{f_{X}(x_{i})}f_{X}(x_{j})\quad\mbox{almost everywhere}$

for every value $x_{j}$ . Consequently,

$\displaystyle f_{Y}(y)$	$\displaystyle=\int f_{YX}(y,~{}x_{j})\,\mbox{d}\mu(x_{j})$	(20)
	$\displaystyle=\int\frac{f_{YX}(y,~{}x_{i})}{f_{X}(x_{i})}f_{X}(x_{j})\,\mbox{d}\mu(x_{j})$	(21)
	$\displaystyle=\frac{f_{YX}(y,~{}x_{i})}{f_{X}(x_{i})}\int f_{X}(x_{j})\,\mbox{d}\mu(x_{j})$	(22)
	$\displaystyle=\frac{f_{YX}(y,~{}x_{i})}{f_{X}(x_{i})}\quad\mbox{almost everywhere},$	(23)

which is equivalent to equation (14), which means that $X$ and $Y$ are statistically independent.
If, on the other hand, $x_{i}$ is such that $\,f_{X}(x_{i})=0$ , then

f_{Y}(y)f_{X}(x_{i})=0

(24)

and

0=f_{X}(x_{i})=\int f_{YX}(y,~{}x_{i})\,\mbox{d}\mu(y)

(25)

implies

f_{YX}(y,~{}x_{i})=0

(26)

for almost every $y$ because the integrand is nonnegative.

∎

Figure 1 shows a case of no correlation between the variables $X$ and $Y$ . One of the reasons for the study of correlation between two variables is to seek a functional relationship between two random variables (See [1] and Figure 1 for examples.). However, when it is not possible to establish a functional relationship between $X$ and $Y$ (see Figure 1 for example), then measuring correlation has not been sufficiently dealt with in the past. Such a situation occurs in the prediction of a surface roughness in a turning operation. The development and application of a correlation concept for such a scenario is the objective of this paper. Existing correlation measures are briefly introduced in next section. Let $\,x_{1},~{}x_{2},~{}\dots,~{}x_{n}\,$ and $\,y_{1},~{}y_{2},~{}\dots,~{}y_{n}\,$ be samples of two random variables $X$ and $Y$ , respectively.

2.1 Pearson Correlation Coefficient

The standard Pearson correlation coefficient [10]

r:=\frac{\sum_{i=1}^{n}(x_{i}-\bar{x})(y_{i}-\bar{y})}{\sqrt{\sum_{i=1}^{n}(x_{i}-\bar{x})^{2}\sum_{i=1}^{n}(x_{i}-\bar{x})^{2}}}

(27)

of Pearson [10], where $\bar{x}$ and $\bar{y}$ are the sample means respectively. The range of correlation coefficient $r$ is [-1, 1]. The closer $r$ is to 1 or -1, the stronger the correlation between the two random variables $X$ and $Y$ . The degree of the linear dependency between $X$ and $Y$ can be measured through $|r|$ : $\,|r|=1\,$ if and only if the points $(x_{i},~{}y_{i})$ describe a straight line in $\mathbb{R}^{2}$ which is neither horizontal nor vertical.

For the random variable $X$ and $Y$ in Fig. 1, $|r|=-0.136$ . Since pearson correlation only detect the linear dependency, so if $X$ and $Y$ are nonlinearly correlated, the pearson correlation is still close to 0. For example, the $r$ for Fig. 1 is 0.015.

2.2 Spearman’s Rank Correlation Coefficient

We present the Spearman’s rank correlation which is a nonparametric correlation coefficient for two numerical variables denoted by $\rho$ , which ranges from -1 to 1 [10]. And the more the $|\rho|$ is closer to 0, the less association between $X$ and $Y$ . First, sort $\,x_{1},~{}x_{2},~{}\dots,~{}x_{n}\,$ and $\,y_{1},~{}y_{2},~{}\dots,~{}y_{n}\,$ in an ascending order. Next, complete a sequence $\,\alpha_{1},~{}\alpha_{2},~{}\dots,~{}\alpha_{n}\,$ in which $\alpha_{i}$ is the position of the corresponding element $x_{i}$ in the sorted sequence i.e. $\alpha_{i}$ = 1 if $x_{i}$ is the smallest , $\alpha_{i}$ = 2 if $x_{i}$ is the second smallest and so on. In a similar fashion, create a sequence $\,\beta_{1},~{}\beta_{2},~{}\dots,~{}\beta_{n}\,$ of ranks corresponding to the sequence $\,y_{1},~{}y_{2},~{}\dots,~{}y_{n}\,$ defined as

\rho=1-\frac{6\sum_{i=1}^{n}(\alpha_{i}-\beta_{i})^{2}}{n(n^{2}-1)}.

(28)

The Spearman’s rank correlation coeffecient $|\rho|$ determines the degree to which a monotonic relationship exists between the two variables $X$ and $Y$ .

Fig. 1 has no apparent monotonic relationship, so its $|\rho|$ = 0.108. Although Fig. 1 has clear nonlinear relationship, but due to the limitation of Spearman’s, the $|\rho|$ is also very small, which is 0.034. Fig. 1 has a rough monotonic increasing relationship, the $|\rho|$ = 0.889, which is close to 1.

2.3 Kendall’s Tau

The Kendall nonparametric measure $\tau$ of correlation between $X$ and $Y$ [5] is defined as

\tau:=\frac{2\sum_{i=1}^{n-1}\sum_{j=i+1}^{n}\Delta_{ij}}{n(n-1)},

(29)

where

\Delta_{ij}:=\left\{\begin{array}[]{cc}1&\text{if}\phantom{a}(x_{j}-x_{i})(y_{j}-y_{i})>0\cr 0&\text{if}\phantom{a}(x_{j}-x_{i})(y_{j}-y_{i})=0\cr-1&\text{if}\phantom{a}(x_{j}-x_{i})(y_{j}-y_{i})<0\end{array}\right.

(30)

If all values in both of the sequences $\,x_{1},~{}x_{2},~{}\dots,~{}x_{n}\,$ and $\,y_{1},~{}y_{2},~{}\dots,~{}y_{n}\,$ are different, then Kendall’s tau for these two sequences is equal to another correlation measure, which is called Goodman and Kruskal’s gamma [3]. The range of $\tau$ is also between [-1, 1], which is similar as Pearson correlation coefficient and Spearman’s rank correlation coefficient. If $|\tau|$ is close to 0, it means very small ordinal association between $X$ and $Y$ is found. Inversely, when $|\tau|$ is close to 1, $X$ and $Y$ have strong ordinal association.

Fig. 1 doesn’t have an obvious ordinal association between $X$ and $Y$ , so its $|\tau|$ is close to 0, $|\tau|=0.070$ . Fig. 1 has ordinal association between $X$ and $Y$ only for the first half observations and an inverse ordinal association for the second half observations, so its $|\tau|$ is also close to 0, $|\tau|=0.033$ . However, Fig. 1 has an obvious ordinal association between $X$ and $Y$ , so its $|\tau|$ is relatively large, which is 0.707.

2.4 Nonlinear Correlation Coefficient

For two variables $X,Y$ , we can estimate its correlation by calculate their mutual information after sorting and grouping their values [11]. Given the discrete variables $X=\{x_{1},x_{2},...,x_{n-1},x_{n}\},Y=\{y_{1},y_{2},...,y_{n-1},y_{n}\}$ :

Step 1 Sort $\{x_{i}\},\{y_{i}\},i=1,2,..,n$ in ascending order. Then, we use $X^{s},Y^{s}$ to represent the ordered $X,Y$ . $X^{s}=\{x_{(1)},x_{(2)},...,x_{(n-1)},x_{(n)}\},Y^{s}=\{y_{(1)},y_{(2)},...,y_{(n-1)},y_{(n)}\}$ , and $x_{(1)}\leq x_{(2)}\leq...\leq x_{(n-1)}\leq x_{(n)},y_{(1)}\leq y_{(2)}\leq...\leq y_{(n-1)}\leq y_{(n)}$

Step 2 Split $X^{s},Y^{s}$ into $b$ ranks , and each rank contains $\frac{n}{b}$ observations.

Step 3 For the pair $\{(x_{(i)},y_{(i)})\},i=1,2,..n,$ we split them into $b^{2}$ regions.

Step 4 Calculate noncorrelation coefficient between $X$ and $Y$ with the mutual information based on the ranks

The nonlinear correlation coefficient (NCC) is as below:

NCC(X;Y)=H(X^{s})+H(Y^{s})-H(X^{s},Y^{s})

, $H(X^{s}),H(Y^{s})$ are the entropy for $X^{s},Y^{s}$ , which are calculated based on $b$ ranks, and $H(X^{s},Y^{s})$ is the joint entropy,which are calculated based on $b^{2}$ regions.

H(X^{s})=-\sum_{i=1}^{b}\frac{n_{i}}{n}\log_{b}\frac{n_{i}}{n}=-\sum_{i=1}^{b}\frac{n/b}{n}\log_{b}\frac{n/b}{n}=1,

H(Y^{s})=-\sum_{j=1}^{b}\frac{n_{j}}{n}\log_{b}\frac{n_{j}}{n}=-\sum_{j=1}^{b}\frac{n/b}{n}\log_{b}\frac{n/b}{n}=1,

and

H(X^{s},Y^{s})=-\sum_{i=1}^{b}\sum_{j=1}^{b}\frac{n_{ij}}{n}\log_{b}\frac{n_{ij}}{n},

$b$ is the base for logarithm, $n_{i},n_{j}$ are the number of observations in $i$ -th, $j$ -th rank, $n_{ij}$ is the number of observations in $(i,j)$ ragion.

Thus,

NCC(X;Y)=2+\sum_{i=1}^{b}\sum_{j=1}^{b}\frac{n_{ij}}{n}\log_{b}\frac{n_{ij}}{n}

The range for this nonlinear correlation coefficient is $[0,1]$ . And 1 means that a strong nonlinear relationship is being detected and 0 means no nonlinear relationship is being found.

We choose $b=10$ for calculating the nonlinear correlation coefficient. If $X$ and $Y$ are not related, the $NCC$ is small (see example Fig. 1, whose $NCC$ is 0.239). $NCC$ for Fig. 1 and Fig. 1 are 0.433 and 0.716, which are larger since these two figures has more clear pattern than Fig. 1. In addition, for Fig. 2, $NCC=0.370$ . And for Fig. 3, $NCC=1$ . We can see from the results that when $X$ and $Y$ has a more clear nonlinear relationship, its $NCC$ is relatively larger.

3 FECHNER CORRELATION COEFFICIENT

In this section Fechner correlation coefficient [6] is reviewed which is not as widely known in the literature.

3.1 Definition and Interpretation

The Fechner correlation coefficient is defined as

\kappa:=\frac{1}{n}\sum_{i=1}^{n}\mbox{sign}(x_{i}-\bar{x})\mbox{sign}(y_{i}-\bar{y}),

(31)

where $\bar{x}$ and $\bar{y}$ are the sample means of the sequences $\,(x_{1},~{}x_{2},~{}\dots,~{}x_{n})\,$ and $\,(y_{1},~{}y_{2},~{}\dots,~{}y_{n})$ , respectively, and

\mbox{sign}(u):=\left\{\begin{array}[]{cc}1&\text{if}\phantom{a}u\geq 0\cr-1&\text{if}\phantom{a}u<0\end{array}\right.

(32)

is the sign function. Fechner correlation coefficient $\kappa$ is calculated using the following scheme:

Step 1 The sequence $\,((x_{1},~{}y_{1}),~{}(x_{2},~{}y_{2}),~{}\dots,~{}(x_{n},~{}y_{n}))\,$ is sorted in an based on $x_{i}$ . Let $i_{0}$ denote the largest index $i$ with $\,x_{i}<\bar{x}\,,1\leq i\leq n$ .

Step 2 The sequence from Step 1 is converted to a binary sequence $\,b=(b_{1},~{}b_{2},~{}\dots,~{}b_{m})\\ (m\leq n)\,$ by replacing an element $\,(x_{i},~{}y_{i})\,$ by 0, if $\,y_{i}<\bar{y}\,$ and by 1, if $\,y_{i}\geq\bar{y}$ .

Step 3 The Fechner correlation coefficient is then calculated as

\kappa=\frac{1}{n}\left(\sum_{i=1}^{i_{0}}(1-2b_{i})+\sum_{i=i_{0}+1}^{n}(2b_{i}-1)\right).

(33)

Note that $\kappa$ = 1 if the sequence b has the form $b=(0,~{}0,~{}\dots,~{}0,~{}1,~{}1,~{}\dots,~{}1)$ with the jump from 0 to 1 occuring at the indices $\,i_{0}+1$ . On the other hand, $\kappa$ is equal to $-1$ if the sequence $b$ has the form $b=(1,~{}1,~{}\dots,~{}1,~{}0,~{}0,~{}\dots,~{}0)$ with the jump from 1 to 0 occuring again at the indices $\,i_{0}+1$ . Figure 2, $\,\kappa=0.907$ . For Figure 1, $\,\kappa=-0.020$ and for Figure 1, the $\kappa$ = 0.580.

As in the case for a straight line associated with the standard Pearson correlation coefficient, $\kappa$ is related to a prediction model. For $\,|\kappa|\approx 1\,$ it provides a classification scheme for classifying the values $y$ given the values $x$

		$\displaystyle y<\bar{y}\quad\text{if}\quad(x-\bar{x})\mbox{sign}(\kappa)<0,$
		$\displaystyle y=\bar{y}\quad\text{if}\quad x=\bar{x},$		(34)
		$\displaystyle y>\bar{y}\quad\text{if}\quad(x-\bar{x})\mbox{sign}(\kappa)>0$

3.2 Properties

One of the drawbacks of the Fechner correlation coefficient is that it does not provide any insight of the shape of the data $\,\{(x_{i},~{}y_{i}):i~{}=~{}1,~{}2,~{}\dots,~{}n\}$ . However demonstrated in Figure 1 and Figure 2, due to the information reduction in Step 1 and 2, $\kappa$ permits the detection of correlations even when accurate predictions of $Y$ are not possible. This can be a big advantage of the Fechner correlation coefficient, at least in certain cases.

Assume that data points $\,(x_{i},~{}y_{i})\,$ lie on a straight line of the form $y=ax+b$ .

If $\,a=0$ , then

\kappa=\frac{1}{n}\sum_{i=1}^{n}\mbox{sign}(x_{i}-\bar{x})\simeq\frac{\frac{n}{2}+(-\frac{n}{2})}{n}=0.

(35)

For $\,a\neq 0\,,$ for $\,i~{}=~{}1,~{}2,~{}\dots,~{}n\,$ it follows that

\bar{y}=\frac{1}{n}\sum_{i=1}^{n}y_{i}=\frac{1}{n}\Bigl{(}a\sum_{i=1}^{n}x_{i}+nb\Bigr{)}=a\bar{x}+b

(36)

consequently the Fechner correlation coefficient becomes

\kappa=\frac{1}{n}\sum_{i=1}^{n}\mbox{sign}(x_{i}-\bar{x})\mbox{sign}[a(x_{i}-\bar{x})]=\frac{1}{n}\sum_{i=1}^{n}a(\mbox{sign}(x_{i}-\bar{x}))^{2}=\mbox{sign}\,a.

(37)

That is $\,|\kappa|=1\,$ if the points $\,(x_{i},~{}y_{i})\phantom{a}\text{for}\phantom{a}i~{}=~{}1,~{}2,~{}\dots,~{}n\,$ lie on a straight line $--$ a property shared with the correlation coefficients.

However, often data can not be a sample from a strictly monotonically increasing function for which $\kappa$ indicates that they are uncorrelated with a small negative correlation. See Figure 3. In the next section,the Fechner correlation coefficient is improved to handle such cases.

4 $g$ -CORRELATION

As discussed earlier, the Fechner correlation coefficient $\kappa$ need not detect monotonic relationships between $X$ and $Y$ as opposed to the correlation measures presented in the subsections 3.B and 3.C. It is shown that the Fechner correlation coefficient can be improved by splitting the data points by a vertical and a horizontal line in a more sensible way instead of arbitrarily dividing the data into 4 classes based on the lines of $\,x=\bar{x}\,$ and $\,y=\bar{y}$

4.1 Definition

As a first step consider the line $\,y=\tilde{y}\,$ , where $\tilde{y}$ is the median of $Y$ , to divide the space of measurements into the following two classes

C_{1}:=\{(x,~{}y)\in\mathbb{R}^{2}~{}|~{}y>\tilde{y}\}\quad\mbox{and}\quad C_{2}:=\{(x,~{}y)\in\mathbb{R}^{2}~{}|~{}y<\tilde{y}\}.

(38)

Assume that the distribution function of $Y$ is continuous and strictly monotonically increasing or decreasing with respect to $X$ .
Case 1: The number of observations of a given dataset, $~{}n$ , is even.

Due to the property of $\tilde{y}$ , we have:

	$\displaystyle{\mbox{P}}(C_{1})$	$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}[\mathbbm{1}\{y_{i}>\tilde{y}\}]=\frac{\frac{n}{2}}{n}=\frac{1}{2}\,,1\leq i\leq n.$		(39)
	$\displaystyle{\mbox{P}}(C_{2})$	$\displaystyle=1-{\mbox{P}}(C_{1})=\frac{1}{2}.$		(39)

And,

\mathbbm{1}\{y>\tilde{y}\}:=\begin{cases}1~{}&{\text{ if }}~{}y>\tilde{y}{}\\ 0~{}&{\text{ if }}~{}y<\tilde{y}{}\end{cases}

(40)

Case 2: The number of observations of a given dataset, $~{}n$ , is odd.

The above assumption means that there is only one data point $(x_{m},y_{m}),~{}m\in[1,n]$ such that $y_{m}=\tilde{y}$ .

Assume $n$ is large and due to the property of $\tilde{y}$ , we have:

	$\displaystyle{\mbox{P}}(C_{1})$	$\displaystyle=\frac{1}{n}\sum_{i=1}^{n}[\mathbbm{1}\{y_{i}>\tilde{y}\}]=\frac{\frac{n-1}{2}}{n}\simeq\frac{1}{2}\,,$		(41)
	$\displaystyle{\mbox{P}}(C_{2})$	$\displaystyle=1-{\mbox{P}}(C_{1})-\frac{1}{n}=\frac{\frac{n-1}{2}}{n}\simeq\frac{1}{2}\,.$		(41)

Thus, each of the classes $C_{1}$ and $C_{2}$ contain about half of the observations in the data set, leading to an optimal separation.

Instead of choosing the fixed line $\,x=\bar{x}\,$ for segmenting the plane that is formed by the dataset into 4 classes $C_{1}^{+},~{}C_{1}^{-},~{}C_{2}^{+},~{}C_{2}^{-}$ , and

$\displaystyle C_{1}^{+}$	$\displaystyle:=\{(x,~{}y)\in\mathbb{R}^{2}~{}\|~{}x>c,~{}y>\tilde{y}\}\quad$	(42)
$\displaystyle C_{1}^{-}$	$\displaystyle:=\{(x,~{}y)\in\mathbb{R}^{2}~{}\|~{}x\leq c,~{}y>\tilde{y}\}\quad$
$\displaystyle C_{2}^{+}$	$\displaystyle:=\{(x,~{}y)\in\mathbb{R}^{2}~{}\|~{}x>c,~{}y<\tilde{y}\}\quad$
$\displaystyle C_{2}^{-}$	$\displaystyle:=\{(x,~{}y)\in\mathbb{R}^{2}~{}\|~{}x\leq c,~{}y<\tilde{y}\},\quad$

we will use the optimum line:

Definition 3 Two random variable $X$ and $Y$ are said to be correlated if there exists $\,c\in\mathbb{R}\,$ such that the criterion

x=c

(43)

assigns realizations $\,(x_{i},~{}y_{i})\,$ of $\,(X,~{}Y)\,$ to class $C_{1}^{+}$ or $C_{1}^{+}$ if they are classified as $C_{1}$ based on equation (38), or class $C_{2}^{+}$ or $C_{2}^{+}$ if they are classified as $C_{2}$ . And the g-correlation is

\operatorname*{argmax}_{c}g(c):=\max\{{\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-}),~{}{\mbox{P}}(~{}C_{1}^{-})+{\mbox{P}}(~{}C_{2}^{+})\}.

(44)

The supremum (the largest upper bound) of all such classification probabilities obtained via the equation (43) for different $c$ is called the $g$ -correlation coefficient of $X$ and $Y$ .

4.2 Properties

Lemma 4 The range of g-correlation coefficient of $X$ and $Y$ is [0.5, 1].

Proof.

For any given $c$ ,

g(c)=\max\{{\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-}),~{}{\mbox{P}}(~{}C_{1}^{-})+{\mbox{P}}(~{}C_{2}^{+})\},

(45)

the restrictions for the above equation are ,

$\displaystyle P(C_{1}^{+})$	$\displaystyle+P(C_{1}^{-})+P(C_{2}^{+})+P(C_{2}^{-})=1$	(46)
$\displaystyle P(C_{1}^{+})$	$\displaystyle+P(C_{1}^{-})=P(C_{1})=0.5$
$\displaystyle P(C_{2}^{+})$	$\displaystyle+P(C_{2}^{-})=P(C_{2})=0.5$
$\displaystyle P(C_{1}^{+})$	$\displaystyle=mP(C_{2}^{+})$
$\displaystyle P(C_{1}^{-})$	$\displaystyle=nP(C_{2}^{-})$

In addition, $m,n>0$ , this is because when $c$ changes, the number of observations on the same side of vertical line $x=c$ will increase or decrease at the same time.

$\displaystyle g(c)$	$\displaystyle=\max\{{\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-}),~{}{\mbox{P}}(~{}C_{1}^{-})+{\mbox{P}}(~{}C_{2}^{+})\}$	(47)
	$\displaystyle=\max\{{\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-}),~{}1-({\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-}))\}$
	$\displaystyle=\max\{m\cdot{\mbox{P}}(~{}C_{2}^{+})+{\mbox{P}}(~{}C_{2}^{-}),~{}1-({\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-}))\}$
	$\displaystyle=\max\{(m-1)\cdot{\mbox{P}}(~{}C_{2}^{+})+{\mbox{P}}(~{}C_{2}^{+})+{\mbox{P}}(~{}C_{2}^{-}),~{}1-({\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-}))\}$
	$\displaystyle=\max\{(m-1)\cdot{\mbox{P}}(~{}C_{2}^{+})+0.5,~{}1-((m-1)\cdot{\mbox{P}}(~{}C_{2}^{+})+0.5)\}$
	$\displaystyle=\max\{(m-1)\cdot{\mbox{P}}(~{}C_{2}^{+})+0.5,~{}(1-m)\cdot{\mbox{P}}(~{}C_{2}^{+})+0.5\}$

In addition, since ${\mbox{P}}(~{}C_{2}^{+})$ is a probability, thus ${\mbox{P}}(~{}C_{2}^{+})\geq 0$ .

When $m-1\geq 0$ ,

	$\displaystyle(m-1)\cdot{\mbox{P}}(~{}C_{2}^{+})+0.5\geq 0.5,$		(48)
	$\displaystyle(1-m)\cdot{\mbox{P}}(~{}C_{2}^{+})+0.5\leq 0.5,$		(48)

and the equation (47) equals to:

g(c)=(m-1)\cdot{\mbox{P}}(~{}C_{2}^{+})+0.5\geq 0.5.

(49)

Similarly, when $(m-1)<0$ and $m>0$ , the equation (47) equals to:

g(c)=(1-m)\cdot{\mbox{P}}(~{}C_{2}^{+})+0.5\geq 0.5.

(50)

Therefore, $g(c)\geq 0.5$ all the time.

And according to equation (46),

{\mbox{P}}(~{}C_{1}^{+})=0

, and

{\mbox{P}}(~{}C_{1}^{-})={\mbox{P}}(~{}C_{2}^{-})=0.5.

Moreover, from equation (46), we can get that

0\leq{\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-})\leq 1,

(51)

thus, the maximum of equation (47) equals to 1 when ${\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-})=1$ and ${\mbox{P}}(~{}C_{1}^{-})+{\mbox{P}}(~{}C_{2}^{+})=0,$ or ${\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-})=0$ and ${\mbox{P}}(~{}C_{1}^{-})+{\mbox{P}}(~{}C_{2}^{+})=1.$

The range for $g(c)$ is $[0.5,~{}1]$ , thus $g$ -correlation coefficient of $X$ and $Y$ ranges from 0.5 to 1.

And when $g$ -correlation is 0.5, it means the $X$ and $Y$ are not correlated. When $g$ -correlation is 1, $X$ and $Y$ are perfectly correlated. ∎

When we define $g$ -correlation in section 4.1, we assume that the distribution function of $Y$ is continuous and strictly monotonically increasing or decreasing with respect to $X$ . But, if we loose the assumption, $g$ -correlation still works. However, we need to remove all the data points $(x_{i},y_{i})$ , who share the same trait: $y_{i}=\tilde{y}$ , after finding the $\tilde{y}$ with original dataset. Then use the new modified dataset to calculate $g$ -correlation.

Note that if the new modified dataset has 0 data points, which means the $Y$ is constant, we don’t need to calculate the $g$ -correlation since $X$ and $Y$ are uncorrelated for sure based on definition 1 in section 2. Similarly, if $X$ is constant and $Y$ varies, $X$ and $Y$ are uncorrelated as well.

The $g$ -correlation coefficient $\omega$ is in general, not symmetric, as shown in Figure 3. In that respect $\omega$ differs from the rest of the correlation coefficients described earlier. With respect to a $g$ -correlation of $X$ and $Y$ in Figure 3, the set

\{(x_{0},~{}y_{0}):x_{0}>c,~{}y_{0}>\tilde{y}\},

(52)

where $\,(x_{0},~{}y_{0})\,$ are realizations of the random vector $\,(X,~{}Y)$ , contains 50% on the average and the set

\{(x_{0},~{}y_{0}):x_{0}\leq c,~{}y_{0}<\tilde{y}\}

(53)

contains 25% of all measurements on the average. Note that $c$ is optimal because moving the line $\,x=c\,$ to the left would just decrease the probability

{\rm P}(X\leq c,~{}Y<\tilde{y})

(54)

and moving the line $\,x=c\,$ to the right would just decrease the probability

{\rm P}(X>c,~{}Y>\tilde{y}).

(55)

The following lemma establishes the main distinction between $\omega$ and the Fechner correlation coefficient:

Lemma 6 Assume $\,Y=f(X)\,$ for a strictly monotonic continuous function $f(\cdot)$ and the mean of $Y$ is $\tilde{y}$ , then $g$ -correlated between $X$ and $Y$ can be obtained when $c=f^{-1}(\tilde{y})$ , which is 1.

Proof.

Without loss of generality let $f(.)$ be a strictly monotonically increasing function. Suppose that $\epsilon>0$ , define $\alpha_{1}$ and $\alpha_{2}$ by

\alpha_{1}:=\sup\{x\in\mathbb{R}:f(x)<\tilde{y}\}+\epsilon\quad\mbox{and}\quad\alpha_{2}:=\inf\{x\in\mathbb{R}:f(x)>\tilde{y}\}-\epsilon.

(56)

Since $\sup\{x\in\mathbb{R}:f(x)<\tilde{y}\}$ is the largest value of $x$ such that $f(x)<\tilde{y}$ , thus,

\displaystyle f(x)\geq\tilde{y},\text{~{}when~{}}x\geq\alpha_{1}\quad\mbox{and}\quad f(x)<\tilde{y},\text{~{}when~{}}x<\alpha_{1}.

(57)

Similarly, $\inf\{x\in\mathbb{R}:f(x)>\tilde{y}\}$ is the smallest value of $x$ such that $f(x)>\tilde{y}$ , so,

f(x)>\tilde{y},\text{~{}when~{}}x>\alpha_{2}\quad\mbox{and}\quad f(x)\leq\tilde{y},\text{~{}when~{}}x\leq\alpha_{2}.

(58)

In addition, if $\,\alpha_{1}>\alpha_{2}$ , then for every $\,\xi\in[\alpha_{2},~{}\alpha_{1}]\,$ , according to equation(57) and equation(58), $f(\xi)>\tilde{y}$ and $f(\xi)<\tilde{y}$ , which is a contradiction.

Similarly, if $\,\alpha_{1}\leq\alpha_{2}$ , then for every $\xi$ such that $\,\alpha_{1}\leq\xi\leq\alpha_{2}\,$ , we have $\tilde{y}\leq f(\xi)\leq\tilde{y}$ , which is $f(\xi)=\tilde{y}$ .

However, since $\,Y=f(X)\,$ for a strictly monotonic function $f(\cdot)$ , so $x$ and $f(x)$ are one-to-one relationship, and there is a unique $x$ to get the median $\tilde{y}$ . Thus,

x=\alpha_{1}=\alpha_{2}\text{~{}and~{}}f(\alpha_{1})=f(\alpha_{2})=\tilde{y}

(59)

Then, we will show that the $g$ -correlation between $X$ and $Y$ is 1 when we set $c=f^{-1}(\tilde{y})$ , which is also $\alpha_{1}$ .

Let’s split the dataset into 4 classes based on equation (42) with two lines: $x=\alpha_{1}$ , $y=\tilde{y}$ . From the equation(59), (57), (58), We know that no points belongs to class $C_{1}^{-}$ or class $C_{2}^{+}$ and all of them belongs either to class $C_{1}^{+}$ or class $C_{2}^{-}$ . Thus, according to definition 3,

g(c)=\max\{{\mbox{P}}(~{}C_{1}^{+})+{\mbox{P}}(~{}C_{2}^{-}),~{}{\mbox{P}}(~{}C_{1}^{-})+{\mbox{P}}(~{}C_{2}^{+})\}=\max\{1,0\}=1

(60)

∎

All correlation coefficients described in this paper are invariant to linear transformations of the form $\,w=av+d\ (a>0,~{}d\in\mathbb{R})\,$ . For the Pearson correlation coefficient the proof is given in [10] and for the other correlation measures, the proofs are straight forward and hence omitted.

4.3 Estimation of $g$ -Correlation

For a given $\,\{(x_{i},~{}y_{i}):i~{}=~{}1,~{}2,~{}\dots,~{}n\}$ of measurements, the $g$ -correlation coefficient $\omega$ can only be estimated as described next. Consider dividing the data set into two subsets: a training set $T$ of size $q$ and an evaluation set $E$ of size $\,(n-q)$ . First, estimates for the separating lines $\,y=\tilde{y}\,$ and $\,x=c\,$ with an appropriate value of $c$ is found based on the training data set $T$ . For the median $\tilde{y}$ of $Y$ , the sample median

\tilde{y}:=\left\{\begin{array}[]{cc}y^{{}^{\prime}}_{\frac{n+1}{2}}&\text{for}\phantom{a}n\phantom{a}\text{odd}\\ \\ \frac{y^{{}^{\prime}}_{\frac{n}{2}}+y^{{}^{\prime}}_{\frac{n+1}{2}}}{2}&\text{for}\phantom{a}n\phantom{a}\text{even},\end{array}\right.

(61)

where $\,(y^{{}^{\prime}}_{1},y^{{}^{\prime}}_{2},~{}\dots,~{}y^{{}^{\prime}}_{n})$ denotes the sequence $\,(y_{1},~{}y_{2},~{}\dots,~{}y_{n})\,$ of the $y$ -values of $T$ sorted in ascending order, is used.

The following algorithm is used to compute a $c$ which gives an optimal classification for the training set $T$ of measurements with respect to the classes $C_{1}^{+},~{}C_{1}^{-},~{}C_{2}^{+},~{}C_{2}^{-}$ defined in (42) see [8] for an alternative method for finding a resonably good value for $c$ .

Step 1 Sort all pairs in the sequence $\,s={(x_{1},~{}y_{1}),~{}(x_{2},~{}y_{2}),~{}\dots,~{}(x_{q},~{}y_{q})}_{q\geq 1}\,$ in ascending order based on the $x$ -values.

Step 2 Consider the arithmetic means of $x$ s’ of all successive pairs in $s$ as possible candidates for $c$ . Start the smallest value $c$ and proceed successively to the highest value.

Step 3 For the first candidate for $c$ , count the number $p_{1}$ of pairs $\,(x_{i},~{}y_{i})\,$ of $s$ with $\,x_{i}\leq c\,$ and $\,y_{i}<\tilde{y}\,$ along with the number $p_{2}$ of pairs with $\,x_{i}>c\,$ and $\,y_{i}>\tilde{y}$ . For all other candidates update $p_{1}$ and $p_{2}$ based on whether the pairs passed since using the previous candidate belong to $C_{1}^{+}$ or $C_{2}^{-}$ .

Step 4 Store the maximum classification percentage $\,\max\{p_{1}+p_{2},~{}q-p_{1}-p_{2}\}/q\,$ achieved for the test dataset $E$ along with the corresponding candidate for $c$ . Go to Step 2.

Finally, $\omega$ is approximated based on the calculated values $\tilde{y}$ and $c$ using Definition 3 for the dataset $T$ .

5 MULTIDIMENSIONAL $g$ -CORRELATION

The multidimensional correlation problem consists in determining whether there exists a correlation between a random vector $\,(X_{1},~{}X_{2},~{}\dots,~{}X_{M})\,$ of independent variables and a single dependent random variable $Y$ . From all of the correlation coefficients described in this article only Pearson correlation coefficient [12] and the $g$ -correlation coefficient $\omega$ can be generalized for the multidimensional situation.

When generalizing the $g$ -correlation coefficient to $M$ independent variables, the line $\,y=\tilde{y}\,$ becomes a hyperplane while the classes $C_{1}$ and $C_{2}$ become halfspaces,respectively. In order to separate the orthogonal projections $\,(x_{1}^{i},~{}x_{2}^{i},~{}\dots,~{}x_{M}^{i})\,$ of a set $\,\{(x_{1}^{i},~{}x_{2}^{i},~{}\dots,~{}x_{M}^{i},~{}y^{i}):i~{}=~{}1,~{}2,~{}\dots,~{}n\}\,$ of measurements onto the $r$ -dimensional space of the independent variables, one cannot use a line similar to $\,x=c\,$ as in equation (43). Instead, a hyperplane (a plane for $\,M=2\,$ and a straight line for $\,M=1$ ) is sought for separating the orthogonal projections $\,(x_{1}^{i},~{}x_{2}^{i},~{}\dots,~{}x_{M}^{i})(i~{}=~{}1,~{}2,~{}\dots,~{}n)\,$ with respect to the classes $C_{1}$ and $C_{2}$ to which the corresponding measurements $\,(x_{1}^{i},~{}x_{2}^{i},~{}\dots,~{}x_{M}^{i},~{}y^{i})\,$ belong. See [8] for further details about the multidimensional $g$ -correlation and its practical application using Fisher linear discriminant functions. Also the multidimensional $g$ -correlation coefficient is directly related to a prediction model which allows inference of $Y$ from the realizations of $\,X_{1},~{}X_{2},~{}\dots,~{}X_{M}$ .

6 CORRELATION COEFFICIENTS COMPARISON

6.1 Comparison on linearly correlated datasets

As so far, we have introduced 5 correlation coefficients from literature and a new nonlinear non-parametric correlation measure method: $g$ -correlation ( $\omega$ ). We run a comparison on 12 different 2-D simulated dataset with unique features to observe the robustness of $\omega$ .

We can visualize the comparison from Fig.4. The $x$ -axis for each plot (from top to bottom) represents Pearson correlation coefficient ( $r$ ), Spearman’s rank correlation coefficient ( $\rho$ ), Kendall’s Tau correlation coefficient ( $\tau$ ), Fechner correlation coefficient ( $\kappa$ ), and Nonlinear correlation coefficient ( $NCC$ ). The $y$ -axis for all plots are $g$ -correlation.

We can see that the top 4 plots share the same bowl shape that when the Pearson correlation coefficient, Spearman’s rank correlation coefficient, Kendall’s Tau correlation coefficient, and Fechner correlation coefficient are between [-1, 0], the $g$ -correlation will decrease when these four correlation coefficient get closer to 0. This shows that $g$ -correlation is robust and correct. Because when these four correlation coefficients get closer to 0, the relationship of the datasets is transforming from negative correlated to non correlated. Thus $g$ -correlation is changing from its maximum, which is 1, to, its minimum, which is 0.5.

Similarly, when the Pearson correlation coefficient, Spearman’s rank correlation coefficient, Kendall’s Tau correlation coefficient, and Fechner correlation coefficient are between [0, 1], the $g$ -correlation will increase when these four correlation coefficient get closer to 1. Again, this proves $g$ -correlation’s robustness. Because when these four correlation coefficient get closer to 1, the two variables in dataset are positively correlated. Thus $g$ -correlation is getting closer to 1.

The last plot is the comparison between Nonlinear correlation coefficient and $g$ -correlation. As we mentioned in section 2.4, the range for NCC is [0,1], thus . We can see from the plot that NCC and $g$ -correlation are monotonic increasing. This also indicate that $g$ -correlation is correct since when NCC is closer to 1, the corresponding dataset should be perfectly correlated, thus $g$ -correlation should also be 1.

Thus, in summary, $g$ -correlation is robust based on the result and analysis from the experiments with 12 datasets and comparison with 5 existing correlation coefficient measurements, which vary from linear correlation coefficient to non-linear correlation coefficient, from parametric correlation coefficient to non-parametric correlation coefficient.

6.2 Comparison on nonlinearly correlated datasets

In section 6.1, we see that $g$ -correlation is consistent with all the existing 5 coefficient correlation. In this section, we will show two examples which $g$ -correlation outperforms than one or more other correlations in capturing the nonlinear relationship between variable $X$ and $Y$ .

In the graph 5, we can see the variable $X$ and $Y$ are nonlinearly correlated and they are also auto-correlated as time series dataset, which means the pattern of the correlation repeat over certain intervals. When we exam its correlation by Pearson correlation coefficient, the result is -0.058, which is contradicted with our observation. Similarly, Spearman’s rank correlation coefficient is -0.061, Kendall’s Tau correlation coefficient is -0.042, and Fechner correlation coefficient is 0, which are all giving an inaccurate result that $X$ and $Y$ in graph 5 is not correlated.

However, $g$ -correlation shows that these two variables are correlated for sure, whose coefficient is 0.71. By Lemma 4 should we know that the further $g$ -correlation coefficient is from 0.5 the more nonlinearly correlated $X$ and $Y$ are. Graph 5 demonstrates the $g$ -correlation. Following the procedure in section 4.3, we get the $g$ -correlation by splitting the dataset with $y=\tilde{y}$ and $x=c=2.85$ .

In the graph 6, we can see that variables $X$ and $Y$ has some nonlinear correlation since the range for possible $y$ varies when $x$ changes. Roughly speaking, the possible $Y$ of $x<c$ is smaller than that of $x\geq c$ . By Definition 1 in section 2, we know that this kind of dataset is correlated. And the $g$ -correlation coefficient for this dataset is 1. Graph 6 demonstrates the $g$ -correlation and the optimal $c=-12.366$ .

However, $(NCC)$ here is 0.363, which means that $NCC$ didn’t detect the complete pattern.

In summary, the ability for $g$ -correlation captures nonlinear and complex relationship between variables is better than the 5 correlation coefficients in literature.

7 CORRELATION COEFFICIENTS AND SURFACE ROUGHNESS ASSESSMENT

Surface roughness is an important quality indicator for products machined with turning, milling, or grinding processes.An implementation of adaptive control schemes requires in-process assessment of surface roughness. Due to the limitations of stylus profilometers, optical techniques, etc., surface roughness is generally measured based on the following three parameters: arithmetic mean roughness ( $R_{a}$ ), maximum peak-to-valley roughness ( $R_{max}$ ), and mean roughness depth ( $R_{z}$ ). We use all the correlation coefficients presented in this paper to determine the correlation between the average level of the three surface roughness parameters which act as the dependent variables, and cutting speed and cutting feed as well as average values of the statistics RMS, absolute energy and ringdown counts of acoustic emission signals which act as the independent variables.

Data for 50 experiments with 25 different operating conditions (varying speed and feed rates) were collected and processed. For computing the $g$ -correlation coefficient the 50 records were randomly divided 10,000 times into a training set $T$ of 30 records and an evaluation set $E$ of 20 records. The arithmetic mean of the g-correlations for the respective evaluation set is taken as $\omega$ .

Table A Novel Nonlinear Non-parametric Correlation Measurement With A Case Study on Surface Roughness in Finish Turning presents the correlation coefficients for the above data sets. For identical measurements,we took the average ranks for these equal values in finding the Spearman’s rank correlation coefficient $\rho$ [10]. Figure 7 shows the graphical representation of the results. Each color represents one correlation coefficient measure method. For each line, each marker is an absolute value of a correlation coefficient between one independent variable and one dependent variable. From left to right, each marker represents the correlation coefficient between cutting speed and $R_{a}$ , cutting speed and $R_{max}$ , cutting speed and $R_{z}$ , cutting feed and $R_{a}$ , cutting feed and $R_{max}$ , cutting feed and $R_{z}$ , one of the acoustic emission statistics RMS and $R_{a}$ , one of the acoustic emission statistics RMS and $R_{max}$ , one of the acoustic emission statistics RMS and $R_{z}$ , absolute energy and $R_{a}$ , absolute energy and $R_{max}$ , absolute energy and $R_{z}$ , ringdown counts and $R_{a}$ , ringdown counts and $R_{max}$ , ringdown counts and $R_{z}$ ,

From figure 7 it is seen that $\omega$ has the same pattern as $\,|r|$ , $\,|\rho|$ , $\,|\tau|$ , $\,|\kappa|$ , and $NCC$ . This result is consistent with the result we got from section 6.1 with simulated dataset. In addition, in 6.1, we use full dataset to calculate the $g$ -correlation and the result shows that the $\omega$ is consistent with $\,r$ , $\,\rho$ , $\,\tau$ , $\,\kappa$ , and $NCC$ . In this section, we use real world dataset to calculate $g$ -correlation by estimating the parameters $\tilde{y}$ and $c$ in training set and validating them in test set. And the result also shows that $\omega$ is consistent with $\,r$ , $\,\rho$ , $\,\tau$ , $\,\kappa$ , $NCC$ . It further indicates that $g$ -correlation is robust.

However, in section 6.2, we showed that $g$ -correlation outperforms when there are some complicated nonlinear relationship between independent and dependent variables. In figure 7, we can see that the correlation coefficients between absolute energy and $R_{a}$ , $R_{max}$ , and $R_{z}$ , as well as ringdown counts of emission signals and $R_{a}$ , $R_{max}$ , and $R_{z}$ are close to 0 based on all the correlation coefficient measurements, except for $NCC$ and $\omega$ . This could be case that the hidden nonlinear relationship is captured by $NCC$ and $\omega$ .

From the standpoint of surface roughness prediction in finish turning, the results imply that cutting feed is strongly while the cutting speed and RMS of acoustic emission signals are moderately correlated with the three roughness parameters. The absolute energy and ringdown counts has nonlinear correlation to surface roughness.

8 CONCLUSIONS

Several correlation coefficients have been examined in this paper, with regard to linearly and nonlinearly correlated dataset. We showed that when dealing with linearly correlated variables, $g$ -correlation coefficient $\omega$ is consistent with Pearson’s $r$ , Spearman’s $\rho$ , Kendall’s $\tau$ , Nonlinear Correlation’s $NCC$ as well as Fechner’s $\kappa$ . When examining more complicated nonlinear relationship, $\omega$ outperforms than all the other 5 measurements.

We also examined these correlation coefficients with regard to a problem of surface roughness assessment in finish turning. It was possible to verify former results about surface roughness prediction such as the usefulness of cutting feed through the whole spectrum of correlation coefficients. In addition, $g$ -correlation is consistent with other correlation measurement methods and it can also detect some complex nonlinear relationship that most of other methods can’t do.

In addition, properties of the $g$ -correlation coefficient $\omega$ have been proven and an algorithm for the computation of $\omega$ has been provided.

What’s more, there is no assumptions on the application of $\omega$ , which makes it a universal correlation coefficient measurement method to capture either linear or nonlinear relationship. This together with the facts that it works beyond functional relationships (no parameter needs to estimate) between the data allows the $g$ -correlation coefficient $\omega$ to be applied in a wide range of areas.

Appendix A APPENDIX

Symbol	Description
$X,Y$	Random variables
$r$	Pearson correlation coefficient
$\tau$	Kendall’s Tau correlation coefficient
$\rho$	Spearman’s rank correlation coefficient
$NCC$	Nonlinear correlation coefficient
$\kappa$	Fechner correlation coefficient
$\omega$	$g$ -correlation correlation coefficient
$\|x\|$	Absolute value of $x$
$\bar{x}$	Mean value of variable $X$
$\tilde{x}$	Median value of variable $X$
$\simeq$	Approximately equal to
$\epsilon$	an arbitrarily very small real number
sup	supremum (least upper bound)
inf	infimum (greatest lower bound)
$:=$	is defined to be equal to
$F_{X},F_{Y}$	Cumulative distribution function
$P\{X<b\}$	Probability that $X$ is strictly less than $b$
$P\{X\|Y\}$	Probability of the event $X$ conditional on the event $Y$
$\sum_{i=1}^{n}a_{i}$	$a_{1}+a_{2}+a_{3}+\cdot\cdot\cdot+a_{n}$
$T$	Training set
$E$	Evaluation set
$c,\alpha_{1},\alpha_{2},\xi$	Some real number
$f(\cdot)$	A function that avoids a dummy variable
$f^{-1}(X)$	The inverse function of function $f(X)$

References

[1] Frances Drake and Yacob Mulugetta. Assessment of solar and wind energy resources in ethiopia. i. solar energy. Solar energy, 57(3):205–217, 1996.
[2] Michael C Fleming et al. Principles of applied statistics rout ledge, london national guidelines and standards for industrial effluents, gaseous, emissions and hazardous waste management in nigeria, 1994.
[3] Linton C Freeman. Elementary applied statistics: for students in behavioral science. John Wiley & Sons, 1965.
[4] RV Hogg and EA Tanis. Sampling distribution theory. Probability and Statistical Inference. Prentice-Hall, Upper Saddle River, NJ, pages 262–3, 1997.
[5] Myles Hollander and Douglas A Wolfe. Nonparametric statistical methods. 1999.
[6] Andrey Latyshev and Petr Koldanov. Investigation of connections between pearson and fechner correlations in market network: Experimental study, Jan 1970.
[7] PE Pfeiffer. Probability for applications. 1990.
[8] Stefan Pittner, Sagar V Kamarthi, Piroj Wongsiripatanakul, and Naken Wongvasu. Correlation between acoustic emission statistics and surface roughness in finish turning. Citeseerx Google Scholar, 2000.
[9] William D Richards. The Zen of empirical research. Hampton Press, 1998.
[10] VK Rohatgi. An introduction to probability theory and mathematical statistics. ed. john wiley & sons. 1976.
[11] Zhiyuan Shen, Qiang Wang, and Yi Shen. A new non-liner correlation measure. In 2009 IEEE Youth Conference on Information, Computing and Telecommunication, pages 11–14, 2009.
[12] George O Wesolowsky. Multiple regression and analysis of variance: An introduction for computer users in management and economics. John Wiley & Sons, 1976.

Refer to caption — Figure 1: (a) Uncorrelated meaurements, (b) curvilinear correlation, (c) correlation for unknown and coars shape a correlation which seems to allow only a coarse estimate of the dependent variable Y

Table 1

Comparison of various correlation coefficients for an independent variable, (one of the cutting speed, cutting feed, or one of the acoustic emission statistics RMS, absolute energy and ringdown counts) and a dependent variable (one of the surface roughness parameters $R_{a}$ , $R_{max}$ , and $R_{z}$ .)

	$R_{a}$						$R_{max}$						$R_{z}$
	$r$	$\rho$	$\tau$	$\kappa$	$NCC$	$\omega$	$r$	$\rho$	$\tau$	$\kappa$	$NCC$	$\omega$	$r$	$\rho$	$\tau$	$\kappa$	$NCC$	$\omega$
Speed	$-0.118$	$-0.205$	$-0.180$	-0.000	0.385	0.699	$-0.475$	$-0.512$	$-0.389$	$-0.240$	0.438	0.741	$-0.365$	$-0.408$	$-0.311$	$-0.160$	0.397	0.716
Feed	0.723	0.713	0.575	0.640	0.656	0.807	0.557	0.557	0.468	0.560	0.593	0.797	0.656	0.634	0.524	0.640	0.632	0.811
RMS	$-0.580$	$-0.563$	$-0.411$	$-0.280$	0.414	0.760	$-0.447$	$-0.374$	$-0.278$	$-0.200$	0.325	0.723	$-0.515$	$-0.474$	$-0.345$	$-0.280$	0.373	0.740
Energy	$-0.129$	$-0.014$	$-0.011$	$-0.120$	0.349	0.655	$-0.151$	$-0.042$	$-0.025$	$-0.040$	0.39	0.661	$-0.124$	$-0.048$	$-0.02$	-0.040	0.373	0.662
Counts	0.015	0.045	0.030	0.120	0.366	0.676	0.108	0.123	0.068	0.120	0.39	0.673	0.085	0.125	0.069	0.120	0.467	0.676

A Novel Nonlinear Non-parametric Correlation Measurement With A Case Study on Surface Roughness in Finish Turning

Abstract

keywords:

1 INTRODUCTION

2 THE CONCEPT OF CORRELATION BETWEEN TWO VARIABLES

Proof.

2.1 Pearson Correlation Coefficient

2.2 Spearman’s Rank Correlation Coefficient

2.3 Kendall’s Tau

2.4 Nonlinear Correlation Coefficient

3 FECHNER CORRELATION COEFFICIENT

3.1 Definition and Interpretation

3.2 Properties

4 gg-CORRELATION

4.1 Definition

4.2 Properties

Proof.

Proof.

4.3 Estimation of gg-Correlation

5 MULTIDIMENSIONAL gg-CORRELATION

6 CORRELATION COEFFICIENTS COMPARISON

6.1 Comparison on linearly correlated datasets

6.2 Comparison on nonlinearly correlated datasets

7 CORRELATION COEFFICIENTS AND SURFACE ROUGHNESS ASSESSMENT

8 CONCLUSIONS

Appendix A APPENDIX

References

4 $g$ -CORRELATION

4.3 Estimation of $g$ -Correlation

5 MULTIDIMENSIONAL $g$ -CORRELATION