Collaborative Machine Learning with Incentive-Aware Model Rewards

Collaborative machine learning (ML) is an appealing paradigm to build high-quality ML models. While an individual party may have limited data, it is possible to build improved, high-quality ML models by training on the aggregated data from many parties. For example, in healthcare, a hospital or healthcare firm whose data diversity and quantity are limited due to its small patient base can draw on data from other hospitals and firms to improve the prediction of some disease progression (e.g., diabetes) (Center for Open Data Enterprise, 2019). This collaboration can be encouraged by a government agency, such as the National Institute of Health in the United States. In precision agriculture, a farmer with limited land area and sensors can combine his collected data with the other farmers to improve the modeling of the effect of various influences (e.g., weather, pest) on his crop yield (Claver, 2019). Such data sharing also benefits other application domains, including real estate in which a property agency can pool together its limited transactional data with that of the other agencies to improve the prediction of property prices (Conway, 2018).

However, any party would have incurred some nontrivial cost to collect its data. So, they would not altruistically donate their data and risk losing their competitive edge. These parties will be motivated to share their data when given enough incentives, such as a guaranteed benefit from the collaboration and a fair higher reward from contributing more valuable data. To this end, we propose to value each party’s contributed data and design an incentive-aware reward scheme to give each party a separate ML model as a reward (in short, model reward) accordingly. We use only model rewards and exclude monetary compensations as (a) in the above-mentioned applications, every party such as a hospital is mainly interested in improving model quality for unlimited future test predictions; (b) there may not be a feasible and clear source of monetary revenue to compensate participants (e.g., due to restrictions, the government may not be able to pay firms using tax revenues); and (c) if parties have to pay to participate in the collaboration, then the total payment is debatable and many may lack funding while still having valuable data.

How then should we value a party’s data and its effect on model quality? To answer this first question, we propose a valuation method based on the informativeness of data. In particular, more informative data induces a greater reduction in the uncertainty of the model parameters, hence improving model quality. In contrast, existing data valuation methods (Ghorbani & Zou, 2019; Ghorbani et al., 2020; Jia et al., 2019a, b; Yoon et al., 2020) measure model quality via its validation accuracy, which calls for a tedious or even impossible process of needing all parties to agree on a common validation dataset. Inaccurate valuation can also arise due to how a party’s test queries, which are likely to change over time, differ from the validation dataset. Our data valuation method does not make any assumption about the current or future distribution of test queries.

Next, how should we design a reward scheme to decide the values of model rewards for incentivizing a collaboration? Intuitively, a party will be motivated to collaborate if it can receive a better ML model than others who have contributed less valuable data, and than what it can build alone or get from collaborating separately with some parties. Also, the parties often like to maximize the total benefit from the collaboration. These incentives appear related to solution concepts (fairness, individual rationality, stability, and group welfare) from cooperative game theory (CGT), respectively. However, as CGT assumptions are restrictive for the uniquely freely replicable¹¹1Data and model reward, like digital goods, can be replicated at zero marginal cost and given to more parties. nature of our model reward, these CGT concepts have to be adapted for defining incentives for our model reward. We then design a novel model reward scheme to provide these incentives.

Finally, how can we realize the values of the model rewards decided by our scheme? How should we modify the model rewards or the data used to train them? An obvious approach to control the values of the model rewards is to select and only train on subsets of the aggregated data. However, this requires considering an exponential number of discrete subsets of data, which is intractable for large datasets such as medical records. We avoid this issue by injecting noise into the aggregated data from multiple parties instead. The value of each party’s model reward can then be realized by simply optimizing the continuous noise variance parameter.

The specific contributions of our work here include:

• $\bullet$

  Proposing a data valuation method using the information gain (IG) on model parameters given the data (Sec. 3);

• $\bullet$

  Defining new conditions for incentives (i.e., Shapley fairness, stability, individual rationality, and group welfare) that are suitable for the freely replicable nature of our model reward (Sec. 4.1). As these incentives cannot be provided all at the same time, we design a novel model reward scheme with an adjustable parameter to trade off between them while maintaining fairness (Sec. 4.2);

• $\bullet$

  Injecting Gaussian noise into the aggregated data from multiple parties and optimizing the noise variance parameter for realizing the values of the model rewards decided by our scheme (Sec. 4.3); and

• $\bullet$

  Demonstrating interesting properties of our model reward scheme empirically and evaluating its performance with synthetic and real-world datasets (Sec. 5).

To the best of our knowledge, our work here is the first to propose a collaborative ML scheme that formally considers incentives beyond fairness and relies solely on model rewards to realize them. Existing works (Jia et al., 2019a, b; Ohrimenko et al., 2019) have only looked at fairness and have to resort to monetary compensations if considered.

In our problem setting, we consider $n$ honest and non-malicious parties, each of whom owns some data and assume the availability of a trusted central party²²2In reality, such a central party can be found in established data sharing platforms like Ocean Protocol (Ocean Protocol Foundation, 2019) and Data Republic (https://datarepublic.com). who aggregates data from these parties, measures the value of their data, and distributes a resulting trained ML model to each party. We first introduce the notations and terminologies used in this work: Let $N\triangleq\{1,\ldots,n\}$ denote a set of $n$ parties. Any subset $C\subseteq N$ is called a coalition of parties. The grand coalition is the set $N$ of all parties. Parties will team up and partition themselves into a coalition structure $CS$. Formally, $CS$ is a set of coalitions such that $\bigcup_{C\in CS}C=N$ and $C\cap C^{\prime}=\emptyset$ for any $C,C^{\prime}\in CS$ and $C\neq C^{\prime}$. The data of party $i\in N$ is represented by $D_{i}\triangleq({\rm{\mathbf{X}}}_{i},{\rm{\mathbf{y}}}_{i})$ where ${\rm{\mathbf{X}}}_{i}$ and ${\rm{\mathbf{y}}}_{i}$ are the input matrix and output vector, respectively. Let $v_{C}$ denote the value of the (aggregated) data $D_{C}\triangleq\{D_{i}\}_{i\in C}$ of any coalition $C\subseteq N$. We use $v_{i}$ to represent $v_{\{i\}}$ to ease notation. For each party $i\in N$, $r_{i}$ denotes the value of its received model reward.

The objective is to design a collaborative ML scheme for the central party to decide and realize the values $(r_{i})_{i\in N}$ of model rewards distributed to parties $1,\ldots,n$. The scheme should satisfy certain incentives (e.g., fairness and stability) to encourage the collaboration. Some works (Ghorbani & Zou, 2019; Jia et al., 2019a, b) have considered similar problems and can fairly partition the (monetary) value $v_{N}$ of the entire aggregated data into $r_{i}$ for $i\in N$. They achieved this using results from cooperative game theory (CGT). Our problem, however, differs and cannot be addressed directly using CGT. This is due to the freely replicable nature of our model reward – the total value $\sum_{i\in N}r_{i}$ of received model rewards over all parties $i\in N$ can exceed $v_{N}$.

We will next show how to assess the value of data (Sec. 3) and, more importantly, how to design the reward scheme to decide the values $(r_{i})_{i\in N}$ of model rewards accordingly and realize these values for achieving our incentive-aware objective (Sec. 4).

A set $D_{C}$ of data is considered more valuable (i.e., higher $v_{C}$) if it can be used to train a higher-quality (hence more valuable) ML model. Existing data valuation methods (Ghorbani & Zou, 2019; Ghorbani et al., 2020; Jia et al., 2019b; Yoon et al., 2020) measure the quality of a trained ML model via its validation accuracy. However, these methods require the central party to carefully select a validation dataset that all parties must agree on. This is often a tedious or even impossible process, especially if every party’s test queries, which are likely to change over time, differ from the validation dataset.³³3In the unlikely event that all parties can agree on a common validation dataset, validation accuracy would be the preferred measure of model quality and hence data valuation method. For example, two private hospitals $\mathcal{H}_{1}$ and $\mathcal{H}_{2}$ aggregate their data for diabetes prediction. $\mathcal{H}_{1}$ and $\mathcal{H}_{2}$ prefer accurate test predictions for female and young patients, respectively. Due to the differences in their data sizes, data qualities, and preferred test queries, it is difficult for the central party to decide the demographics of the patients in the validation dataset such that the data valuation is unbiased and accurate.

To circumvent the above-mentioned issues, our proposed data valuation method instead considers an information-theoretic measure of the quality of a trained model in terms of the reduction in uncertainty of the model parameters, denoted by vector $\boldsymbol{\theta}$, after training on data $D_{C}$. We use the prior entropy $\mathbb{H}(\boldsymbol{\theta})$ and posterior entropy $\mathbb{H}(\boldsymbol{\theta}|D_{C})$ to represent the uncertainty of $\boldsymbol{\theta}$ before and after training on $D_{C}$, respectively. So, if the data $D_{C}$ for $C\subseteq N$ can induce a greater reduction in the uncertainty/entropy of $\boldsymbol{\theta}$ or, equivalently, information gain (IG) $\mathbb{I}(\boldsymbol{\theta};D_{C})$ on $\boldsymbol{\theta}$:

then a higher-quality (hence more valuable) model can be trained using this more valuable/informative data $D_{C}$. IG is an appropriate data valuation method as it is often used as a surrogate measure of the test/predictive accuracy of a trained model (Krause & Guestrin, 2007; Kruschke, 2008) since the test queries are usually not known a priori. We empirically demonstrate such a surrogate effect in Appendix F.3. The predictive distribution of the output $y^{*}$ at the test input ${\rm{\mathbf{x}}}^{*}$ given data $D_{C}$ is calculated by averaging over all possible model parameters $\boldsymbol{\theta}$ weighted by their posterior belief $p(\boldsymbol{\theta}|D_{C})$, i.e., $p(y^{*}|{\rm{\mathbf{x}}}^{*},D_{C})=\int p(y^{*}|{\rm{\mathbf{x}}}^{*},\boldsymbol{\theta})\ p(\boldsymbol{\theta}|D_{C})\ \text{d}\boldsymbol{\theta}.$ By reducing the uncertainty in $\boldsymbol{\theta}$, we can further rule out models that are unlikely given $D_{C}$ and place higher weights (i.e., by increasing $p(\boldsymbol{\theta}|D_{C})$) on models that are closer to the true model parameters, thus improving the predictive accuracy for any test query in expectation. The value $v_{C}$ (1) of data $D_{C}$ has the following properties:

• $\bullet$

  Data of an empty coalition has no value: $v_{\emptyset}=0\ .$

• $\bullet$

  Data of any coalition $C\subseteq N$ has non-negative value: $\forall C\subseteq N\ \ v_{C}\geq 0\ .$

• $\bullet$

  Monotonicity. Adding more parties to a coalition cannot decrease the value of its data: $\forall C\subseteq C^{\prime}\subseteq N\ \ v_{C^{\prime}}\geq v_{C}\ .$

• $\bullet$

  Submodularity. Data of any party $i$ is less valuable to a larger coalition which has more parties and data:

$\forall i\in N\ \ \forall C\subseteq C^{\prime}\subseteq N\setminus\{i\}\ \ v_{C^{\prime}\cup\{i\}}-v_{C^{\prime}}\leq v_{C\cup\{i\}}-v_{C}\ .$

The second and third properties are due to the “information never hurts” bound for entropy (Cover & Thomas, 1991). The submodular property of IG (1) assumes conditional independence of the data $D_{i}$ and $D_{j}$ given $\boldsymbol{\theta}$ for any $i,j\in N$ and $i\neq j$; its proof is in Appendix A.

The first two properties fulfill standard assumptions of CGT. The latter two properties will influence the design and properties of our model reward scheme in Sec. 4. For example, the monotonic property ensures that the value $r_{i}$ of any party $i$’s model reward is never negative (Sec. 4).

Recall our problem setting in Sec. 2 that the central party will train a model for each party as a reward. The reward should be decided based on the value of each party’s data relative to that of the other parties’. Let $D_{i}^{r}$ be the data (i.e., derived from $D_{N}$) used to train party $i$’s model reward. The value of party $i$’s model reward is $r_{i}\triangleq\mathbb{I}(\boldsymbol{\theta};D_{i}^{r})$ according to Sec. 3. In this section, we will first present the incentives to encourage collaboration (Sec. 4.1), then describe how our incentive-aware reward scheme will satisfy them (Sec. 4.2), and finally show how to vary $D_{i}^{r}$ to realize the values of the model rewards decided by our scheme (Sec. 4.3).

We desire a collaboration involving the grand coalition (i.e., $CS=\{N\}$) as it results in the largest aggregated data $D_{N}$ and hence allows the highest value of model reward to be achieved, which eliminates the tedious process of deciding how the parties should team up and partition themselves. In Sec. 4.1.2, we will discuss how to incentivize the grand coalition formation and increase the total benefit from the collaboration.

This section empirically evaluates the performance and properties of our reward scheme (Sec. 4.2) on Bayesian regression models with the (a) synthetic Friedman dataset with $6$ input features (Friedman, 1991), (b) diabetes progression (DiaP) dataset on the diabetes progression of $442$ patients with $9$ input features (Efron et al., 2004), and (c) Californian housing (CaliH) dataset on the value of $20640$ houses with $8$ input features (Pace & Barry, 1997). We use a Gaussian likelihood and assume that the model hyperparameters are known or learned using maximum likelihood estimation.

The performance of our reward scheme is evaluated using the IG and mean negative log probability (MNLP) metrics:

where $D^{*}$ denotes a test dataset, and $\mu_{*}$ and $\sigma_{*}^{2}$ denote, respectively, the predictive mean and variance of the predictive distribution $p(y^{*}|{\rm{\mathbf{x}}}^{*},D_{i}^{r})$. We then use these metrics to show how an improvement in IG can affect the predictive accuracy of a trained model on a common test dataset.⁹⁹9We use a randomly sampled test dataset to illustrate how IG is a suitable surrogate measure of the test/predictive accuracy of a trained model. In Sec. 3, we have discussed that in practice, it is often tedious or even impossible for all parties to agree on a common validation dataset, let alone a common set of test queries. More details of the experimental settings such as how to select the test dataset are in Appendix E. The performance of our reward scheme cannot be directly compared against that of the existing works (Ghorbani & Zou, 2019; Jia et al., 2019b) as they mainly focus on non-Bayesian classification models and assume monetary compensations. For all experiments, we consider a collaboration of $n=3$ parties as it suffices to show interesting results.

Gaussian process (GP) regression with synthetic Friedman dataset. For each party, we generate $250$ data points from the Friedman function. We assign party $1$ the most valuable data by spanning the first input feature over the entire input domain $[0,1]$. In contrast, for parties $2$ and $3$, the same input feature spans only the non-overlapping ranges of $[0,0.5]$ and $[0.5,1]$, respectively. This makes the data of parties $2$ and $3$ highly valuable to each other, i.e., $v_{\{2,3\}}$ is high relative to $v_{2}$. Using (2), the Shapley values of the three parties are $\phi_{1}=34.57$, $\phi_{2}=29.24$, and $\phi_{3}=30.78$. Party $1$ has the largest $\phi_{1}$ and will always receive the most valuable model reward with the highest IG or value $v_{N}$.

Fig. 2a shows that our reward scheme indeed satisfies fairness (R5): A party $i$ with a larger $\phi_{i}$ always receives a higher $r_{i}$ (i.e., more valuable model reward). Also, since $\rho_{r}\geq 1$ here, our reward scheme always satisfies individual rationality (R4): Every party $i$ receives a more valuable model reward than that trained on its own data, i.e., its IG $r_{i}$ is higher than $v_{i}$ (see dotted lines in Fig. 2a). As $\rho$ decreases (rightwards), party $1$’s most valuable model reward is unaffected but it has to be “altruistic” to parties $2$ and $3$ with increasing $r_{2}$ and $r_{3}$ (i.e., increasingly valuable model rewards) but smaller $\phi_{2}$ and $\phi_{3}$, as discussed in the last paragraph of Sec. 4.2. When $\rho$ decreases to $\rho_{s}$, stability (R6) is reached: Party $3$’s model reward matches what it will receive by only collaborating with party $2$. When $\rho=0$, all parties receive an equally valuable model reward with the same IG/value $v_{N}$ despite their varying expected marginal contributions. This shows how $\rho$ can be reduced to trade off proportionality in pure Shapley fairness for satisfying other incentives such as stability (R6) and group welfare (R7).

In Fig. 2b, we report the averaged MNLP and shade the $95\%$ confidence interval over $20$ different realizations of ${\rm{\mathbf{z}}}_{i}$. As $\rho$ decreases and each party receives an increasingly valuable model reward (i.e., increasing IG/$r_{i}$), MNLP decreases, thus showing that an improvement in IG translates to a higher predictive accuracy of its trained model.

GP regression with DiaP dataset. We train a GP regression model with a composite kernel (i.e., squared exponential kernel $+$ exponential kernel) and partition the training dataset among $n=3$ parties, as detailed later. The results are shown in Fig. 3.

Neural network regression with CaliH dataset. We consider Bayesian linear regression (BLR) on the last layer of a neural network (NN). $60\%$ of the CaliH data is designated as the “public” dataset¹⁰¹⁰10 In practice, the “public” dataset can be provided by the government to kickstart the collaborative ML. Alternatively, it can be historical data that companies are more willing to share. and used to pretrain a NN. Since the correlation of the house value with the input features in the data of the parties may differ from that in the “public” dataset, we perform transfer learning and selectively retrain only the last layer using BLR with a standard Gaussian prior. So, IG is only computed based on BLR. From the remaining data, we select $400$ data points for the test dataset and $1600$ data points¹¹¹¹11We restrict the total number of data points so that every individual party cannot train a high-quality model alone. for the training dataset to be partitioned among $3$ parties, as detailed later. The results are shown in Fig. 5.

Sparse GP regression with synthetic Friedman dataset (10 parties). We also evaluate the performance of our reward scheme on the collaboration between a larger number $n=10$ of parties. We consider a larger synthetic Friedman dataset of size $5000$ and partition the training dataset among $10$ parties such that each party owns at least $5\%$ of the dataset. We train a sparse GP model based on the deterministic training conditional approximation (Hoang et al., 2015, 2016) and a squared exponential kernel. Since the exact Shapley value (2) is expensive to evaluate with a large number of parties, we approximate it using the simple random sampling algorithm in (Maleki et al., 2013). The results are shown in Fig. 6.

Partitioning Algorithm. To divide any training dataset among parties and vary their contributed data, we first choose an input feature $a$ uniformly at random from all the input features. Next, the data point $d$ is chosen uniformly at random. We partially sort the dataset and divide it into continuous blocks of random sizes, as detailed in Fig. 4.

Such a setup allows parties to have different quantities of unique or partially overlapping data. The input feature $a$ may have real-world significance: If $a$ is the age of patients, then this models the scenario where hospitals have data of patients with different demographics.

Summary of Observations. For any dataset, we consider different train-test splits and partitions of the training dataset using the above algorithm. For each partition, we compute the optimized noise variance $\eta_{i}$ to realize $r^{*}_{i}$ and consider $5$ realizations of ${\rm{\mathbf{z}}}_{i}$ for each party $i\in N$. For a given partition and choice of $\rho$, if the values $(r_{i})_{i\in N}$ of model rewards satisfy individual rationality (R4), then we compute the ratio $\phi_{i}/\phi^{*}$, improvement in IG ($r_{i}-v_{i}$) and MNLP, and maximal (possible) improvement in IG ($v_{N}-v_{i}$) and MNLP for each party $i\in N$. The improvement is measured relative to a model trained only on party $i$’s data. The best/lowest MNLP $m_{N}$ is assumed to be achieved by a model trained on the aggregated data of all parties. Each realization of ${\rm{\mathbf{z}}}_{i}$ (from each party $i$) will produce a point in the scatter graphs in Figs. 3, 5, and 6. If a point lies on the diagonal identity line, then the corresponding party (e.g., with the largest $\phi_{i}$, i.e., $\phi^{*}$) receives a model reward with MNLP $m_{N}$. We also report the number of points with positive improvement in IG and MNLP in the top left hand corner of each graph.

In Figs. 3c-d, 5c-d, and 6c-d, the improvement in MNLP is usually positive: The predictive performance of each party benefits from the collaboration. Occasionally but reasonably, this does not hold when the maximal improvement in MNLP is extremely small or even negative. Lighter colored points lie closer to the diagonal line: Parties with larger $\phi_{i}$ receive more valuable model rewards (Figs. 3a-b, 5a-b, 6a-b) which translates to lower MNLP (Figs. 3c-d, 5c-d, 6c-d).

In Figs. 3, 5 and 6, when $\rho$ decreases from $1$ to $0.5$, darker colored points move closer to the diagonal line, which implies that parties with smaller $\phi_{i}$ can now receive more valuable model rewards with higher predictive accuracy. The number of points (reported in the top left corner) also increase as more of them satisfy R4.

In Fig. 6d with $\rho=0.5$, most points are close to the diagonal identity line because it may be the case that not all training data points are needed to achieve the lowest MNLP. Instead, fewer or noisier data points are sufficient. In this scenario, a larger $\rho$ may be preferred to reward the parties differently. More experimental results for different Bayesian models and datasets are shown in Appendix F.1.

Limitations. Though IG is a suitable surrogate measure of the predictive accuracy of a trained model, a higher IG does not always translate to lower MNLP. In Figs. 3c-d and 5c-d, occasionally, the improvement in MNLP is negative (around $15\%$ of the time) and darker points lie above the diagonal line. The latter suggests that parties with smaller $\phi_{i}$ may receive model rewards with MNLP lower than $m_{N}$ (i.e., awarded to the party with largest $\phi_{i}$, i.e., $\phi^{*}$). This may be purely due to randomness, but we have also investigated factors (e.g., model selection) leading to a weak relationship between IG and MNLP and reported the results in Appendix F.2. Our reward scheme works best when a suitable model is selected and the model prior is not sufficiently informative for any party to achieve a high predictive accuracy by training a model on its own data. In practice, this is reasonable as collaboration precisely happens only when every individual party cannot train a high-quality model alone but can do so from working together.

This paper describes a collaborative ML scheme that distributes only model rewards to the parties. We perform data valuation based on the IG on the model parameters given the data and compute each party’s marginal contribution using the Shapley value. To decide the appropriate rewards to incentivize the collaboration, we adapt solution concepts from CGT (i.e., fairness, Shapley fairness, stability, individual rationality, and group welfare) for our freely replicable model rewards. We propose a novel reward scheme (Theorem 1) with an adjustable parameter that can trade off between these incentives while maintaining fairness. Empirical evaluations show that a smaller $\rho$ and more valuable model rewards translate to higher predictive accuracy. Current efforts on designing incentive mechanisms for federated learning can build upon our work presented in this paper. For future work, we plan to address privacy preservation in our reward scheme. The noise injection method used for realizing the model rewards in this work is closely related to the Gaussian mechanism of differential privacy (Dwork & Roth, 2014). This motivates us to explore how the injected noise will affect privacy in the model rewards.

This research/project is supported by the National Research Foundation, Singapore under its Strategic Capability Research Centres Funding Initiative and its AI Singapore Programme (Award Number: AISG-GC-$2019$-$002$). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore.