链接：https://pan.baidu.com/s/1lopSDMINWqtmEsgYj03Fug 

提取码：2nwt 

 

2 FRAMEWORK AND PROBLEM

2.1 Data Model

Crowdsourcing Tasks: A platform is designed to crowd￾source tasks, deployed by a set of requesters and undertaken

by crowd workers. We consider collaborative tasks such as

sentence translation, text summarization, and puzzle solv￾ing [29, 30].

Deployment Strategies: A deployment strategy [17]

instantiates three dimensions: Structure (sequential or simul￾taneous), Organization (collaborative or independent), and

Style (crowd-only or crowd and algorithms). We rely on com￾mon deployment strategies [5, 17] and refer to them as S.

Figure 2 enlists some strategies that are suitable for text

translation tasks (from English to French in this example).

For instance, SEQ-IND-CRO in Figure 2(a) dictates that work￾ers complete tasks sequentially (SEQ), independently (IND)

and with no help from algorithms (CRO). In SIM-COL-CRO

(Figure 2(b)), workers are solicited in parallel (SIM) to com￾plete a task collaboratively (COL) and with no help from

algorithms (CRO). The last strategy SIM-IND-HYB dictates a

hybrid work style (HYB) where workers are combined with

algorithms, for instance with Google Translate.

Research 1: Crowdsourcing and Visualization SIGMOD ’20, June 14–19, 2020, Portland, OR, USA 5

A platform could provide the ability to implement some

strategies. For instance, communication between workers

enables SEQ, while collaboration enables COL. Additionally,

coordination between machines and humans may enable

HYB. Therefore, strategies could be implemented inside or

outside platforms. In the latter, a platform could be used

solely for hiring workers who are then redirected to an environment where strategies are implemented. In all cases, we

will assume a set of strategies S for a given platform.

For the purpose of illustration, we will only use a few

strategies in this paper. However, in principle, the number

of possible strategies could be very large. The closest analogy is query plans in relational databases in which joins,

selections, and projections could be combined any number

of times and in dierent orders. Additionally, there exists

multiple real world tools Turkomatic [19] or Soylent [4], that

aid requesters in planning and solving collaborative tasks. In

Turkomatic, while workers decompose and solve tasks, requesters can view the status of worker-designed workows

in real time; intervene to change tasks; and request new

solutions. Such tools would certainly benet from strategy

recommendation.

Task Requests and Deployment Parameters:A requester

intends to nd one or more strategies (notationally k, a small

integer) for a deployment d with parameters on quality, cost,

and latency (d.quality, d.cost, d.latency) such that, when a

task in d is deployed using strategy s ∈ S, it is estimated to

achieve a crowd contribution quality s.quality, by spending

at mosts.cost, and the deployment will last at mosts.latency.

Quality Cost Latency

d1 0.4 0.17 0.28

d2 0.8 0.2 0.28

d3 0.7 0.83 0.28

s1 0.5 0.25 0.28

s2 0.75 0.33 0.28

s3 0.8 0.5 0.14

s4 0.88 0.58 0.14

Table 1: Deployment Requests and Strategies

Example 1. Assume there are 3 (m = 3) task deployment

requests for dierent types of collaborative sentence translation

tasks. The rst requester d1 is interested in deploying sentence

translation tasks for 2 days (out of 7 days), at a cost up to $100

(out of $600 max), and expects the quality of the translation to

reach at least 40% of domain expert quality. Table 1 presents

these after normalization between [0 1]. We set k = 3.

A strategy s is suitable to be recommended to d, if

s.quality ≥ d.quality AND s.cost ≤ d.cost AND s.latency ≤ d.latency. Estimating the parameters s.quality, s.cost, s.latency for each s and deployment d requires accounting

for the worker pool and their skills who are available to undertake tasks in d. A simple yet reasonable approach to that is

to rst match task types in a deployment request with workers’

skills to select a pool of workers. Following that, we account for

worker availability from this selected pool, since the deployed

tasks are to be done by those workers. Thus, the (estimated)

quality, cost and latency of a strategy for a task is a function

of worker availability, considering a selected pool of workers

who are suitable for the tasks.

Worker Availability: Worker availability is a discrete

random variable and is represented by its corresponding

distribution function (pdf), which gives the probability of

the proportion of workers who are suitable and available

to undertake tasks of a certain type within a specied time

d.latency (refer to Example 1). This pdf is computed from historical data on workers’ arrival and departure on a platform.

StratRec computes the expected value of this pdf to represent

the available workforce W , as a normalized value in [0, 1].

In the remainder of the paper, worker availability stands for

worker availability in expectation, unless otherwise speci-

ed. How to accurately estimate worker availability is an

interesting yet orthogonal problem and not our focus here.