A Blockchain Protocol for Human-in-the-Loop AI

Modern Artificial Intelligence tends to focus on centralization, autonomy and competition with humans [1]. However, the idea of augmenting human intelligence [2] and "man-computer symbiosis" [3] was prevalent in the early days of AI and cybernetics.

Human-in-the-loop (HITL) machine learning [4] is a promising development in this regard. Intelligent human inputs are often included in the machine learning workflow before training, in the form of data annotation. The HITL approach extends the scope of this integration to include human-machine interactions during training, e.g. through expert supervision [5], and post-training, e.g. in safety audits and fine-tuning models [6].

Software applications for crowdsourcing human intelligence tasks face challenges pertaining to the unfair compensation of labor [7], fraud [8], censorship [9], and the difficulty of vetting credentials [10]. The latter is typified by protocols geared towards centralized crowd-labor platforms, such as Turkit [11]. For example, human input is taken from an indistinct mass of crowd workers on Amazon’s Mechanical Turk platform, without the ability to require a certain level of expertise or credentials from respondents. Turkit [11] introduced the useful concept of scripting human intelligence tasks within traditional web applications, and was designed with issues related to high-cost and high-latency steps involving humans in mind. This required engineering a crash-rerun approach to avoid re-executing expensive steps.

Decentralized software deployed on public, permissionless blockchains offers natural opportunities to tackle the aforementioned challenges. Any write instruction in a smart contract is an atomic transaction that is immutably stored on the blockchain, and transparently accessible to any client application. Moreover, in addition to trustless, uncensorable payment processing, blockchain software can offer participants ownership in the system they partake in. Lastly, the emergence of standards for identity management, such as the non-fungible token standard, allow for sophisticated access control and have propelled the emergence of domain-expert decentralized autonomous organizations (DAOs).

DAOs are sometimes imagined as being governed by autonomous algorithms, with humans at the margins. However, there is an increasing push towards a future of collective intelligence that promotes harmony between humans and algorithms by optimizing for the autonomy of individuals [12]. We believe that protocols that facilitate crowdsourcing of human intelligence and preferences are a key component of this. This has applications in collection and annotation of training data, and AI safety.

In the following, we describe a protocol for Ethereum Virtual Machine (EVM)-compatible blockchains that allow for the on-chain modeling of human intelligence tasks and their integration in machine learning workflows.

A Human Intelligence Primitive (HIP) is a procedure for the collection and representation of preferences structures on a finite set of $n$ potential alternatives (i.e. comparable objects or actions) $A=\{a_{0},\dots,a_{n-1}\}$. Preferences can be of one of the four following types, based on [13]:

• •

  A choice $(P_{1})$, that is a subset $A^{\prime}\subseteq A$ of potential alternatives, typically a singleton set, containing the preferred alternative(s).

• •

  A ranking $(P_{2})$, that is a total preorder on $A$, ordering alternatives by decreasing preference, with possible ex-aequo. A particular case of ranking is preferential voting [18], in which this preference structure is a total order on $A$.

• •

  A sorting $(P_{3})$, that is the assignment of each alternative in $A$ into pre-defined classes $C=\{c_{0},\dots,c_{k-1}\}$, ordered by decreasing preference. A particular case of sorting is score voting [19] on a discrete scale.

• •

  A classification $(P_{4})$, that is the assignment of each alternative in $A$ into pre-defined, unordered classes $C=\{c_{0},\dots,c_{k-1}\}$.

A HIP can thus be abstractly characterized by a triplet $(t,n,k)$, where $t\in\{P_{1},P_{2},P_{3},P_{4}\}$ is the type of preferences sought, $n\in\{2,3,\dots,+\infty\}$ the number of alternatives considered, and $m\in\{1,2,\dots,+\infty\}$ the number of classes (equal to $1$ for a choice or ranking primitive).

We propose a smart contract implementing HIPs to incentivize and coordinate collective intelligence by humans within DAOs. HIPs can be initiated by Externally-Owned Account (EOA) on Ethereum, or contracts, through a CALL or DELEGATECALL operation. Conversely, the smart contract can communicate synchronous events to off-chain clients (e.g. the submission of a response to a HIP), and its output can be read asynchronously by these programs.

We consider two categories of users of the smart contract; proposers and respondents. A HIP is recorded in a HIP object, the creation of which is initiated by a proposer address, through a function submitHIP(). In this function, a proposer submits a triplet $(t,n,k)$, and pays a fee that depends on the type of primitive $t$. The type $t$ is encoded by an enumerable types {CHOICE, RANKING, SORTING, CLASSIFICATION}, while the number of alternatives $n$ and the number of classes $k$ are stored in unsigned integers.

Additionally, the proposer specifies a duration, encoded as an unsigned integer number of seconds, for taking responses. This duration is relative to the creation date of the HIP, recorded as the timestamp of the valid block enacting its creation on the blockchain. Lastly, the HIP object records the number of responses (individual preference structures compatible with the HIP type) recorded so far, in an unsigned integer variable numResponses.

An individual response to a HIP is submitted by a respondent address, through a function submitResponse(), specifying the address of the proposer, and the index of the HIP being responded to, among their proposed HIPs. Respondents’ access is gated by a non-fungible token (NFT), vetting their credentials, and giving them read access to the corresponding off-chain semantic data, and write access to record a response to a HIP in the contract. Moreover, before recording a response, we verify that the respondent has not already voted, and that the submitted response is compatible with the HIP type $t$. An individual response that passes these checks is recorded in a Response structure, containing the address of the respondent and an array of unsigned integers, representing the content of the response.

Given the strengths of the blockchain (trustless access control and payment processing), and its weaknesses (inability to store secrets and high cost of computation), we have made the following key architecture choices:

• •

  Since data are transparently stored on the blockchain, HIP objects are recorded in the abstract form of a triplet $(t,n,k)$, and linked with semantic data (i.e. descriptions for alternatives and classes) that are stored off-chain.

• •

  In order to incentivize responses, and discourage their concentration in a few HIPs, the reward of each respondent is the fee paid by the proposer divided by the number of responses, by the end of a HIP’s duration.

• •

  Once recorded in the contract, responses can be eventually accessed by off-chain clients for computationally complex processing and aggregation.

A wide range of tasks requiring human intelligence can be expressed as HIPs, for example within the training and operation of AI systems and the governance of blockchain systems and DAOs. Surveys can be modeled as an instance of $P_{1}$ [14], the collection of training data for machine-learned ranking (MLR) as $P_{2}$ [15], independent AI safety audits as $P_{3}$ [6], or data annotation as $P_{4}$ [16]. In these examples, HIPs are used with a descriptive intent and a collection of individual preferences is their intended output. Moreover, when combined with a systematic aggregation procedure for individual preferences, HIPs can serve as primitives in processes such as plurality voting or approval voting as instances of $P_{1}$ [17], preferential voting as $P_{2}$ [18], score voting as $P_{3}$ [19], or rule-based classification as $P_{4}$ [20].

Following is an example, in pseudo-JavaScript, for a data annotation use case. The prefix "contract." indicates a call to a function of the contract by an EOA or a web client, e.g. using the web3.js library [21].

• •

  Proposer calls a classification HIP with await contract.methods.submitHIP(CLASSIFICATION, n, 2, duration).send({from:accounts[0], value:fee});

• •

  After a delay corresponding to the value of the argument duration, proposer collects responses with response=await contract.methods.getResponse(proposer,index, i).call();

• •

  Proposer aggregates responses off-chain using e.g. the majority rule.

This paper described the design and implementation of an Ethereum protocol to to incentivize and coordinate collective intelligence by humans on-chain. Experiments using the proposed protocol will be conducted in the Algovera community, a DAO for data scientists, in order to identify new use cases and optimize gas usage for typical real-world applications.

The detailed source code of the proposed implementation can be found in the Appendix of this paper.

This publication has emanated from research conducted with the partial financial support of Algovera Grants under grant number 22/AG/R1/6. The first author is grateful to the members of Algovera DAO for fruitful discussions.