The pitfalls of crowd-sourcing: We might not like what the crowd tells us

I am excited to see more and more initiatives that tap into the wisdom of corporate or public crowds to shape priorities for specific policy agendas.

Just recently, different units in my organization called on staff to submit ideas how to improve their business processes, and let all colleagues vote on them to determine which idea would get most traction and support. This is a great way to improve organizational efficiency while involving those affected by change in an actual change process.

At the same time, the success of the Rio Dialogues showed how crowdsourcing policy recommendations and public voting on them can increase legitimacy of inter-governmental negotiations. This new model of public engagement during a United Nations summit received praise from participants as the “the most inclusive process in the history of global summits” (Josette Sheeran, VP of the World Economic Forum), and opened the door for similar approaches in defining the successors of the Milliennium Development in the Post-2015 process (see also Jamie Drummonds TED talk on crowdsourcing the Post-2015 agenda).

I find this new momentum towards open government and democratization of political as well as organizational processes extremely encouraging. However, while embarking on this journey towards bottom-up participation and openness, it is important that we understand exactly the conditions under which this can work - and under which it might not.

Why is it a good thing to open up agenda setting to crowd-sourced suggestions and voting?

Let me highlight a few aspects that are of importance in particular when using approaches that involve crowdsourced suggestions and a system of voting on those suggestions  in order to identify priorities.

First, why is it a good thing to open up agenda setting to crowdsourced suggestions and voting? Here are what I consider the most important benefits:

1. Participation: By giving people a voice, they get engaged in the process, reflect on issues and contribute to them, rather than being on the mere recipient end of a process.
2. Innovation: By providing an open space for any kind of contribution, the crowd often comes up with out-of-the box ideas that an expert group might otherwise have never thought of.
3. Prioritization: By giving people a vote, we are finding out what is most important to them, vs. what might be most important to those who have the task of setting an agenda.
4. Buy-in: By appreciating the vote of the crowd on following up on suggestions that most people identify with, we create buy-in for the process and increase the likelihood of successful implementation.

The benefits of point 1 and 2 are relatively easy to achieve. The 1.3 million public votes that were received on the recommendations of the Rio Dialogues clearly showed that the public was willing to provide their opinion, and also the call for ideas among staff receives ample attention. People usually really like to be asked for their opinion.

If you ask people for their opinion, you have to mean it

Points 3 and 4 are much more difficult to manage, because - as with any democratic mechanism - asking for a vote does not always mean we are fond of what receives the most votes. And arguably, neither an organization that seeks for input on its own agenda, nor the heads of state within an inter-governmental process are technically obliged to follow the vote of the constituencies they asked for their opinions. However, if one has chosen to go down the path of asking people for their input, there is a certain responsibility that comes with it. And once we go into prioritization of specific suggestions over others, there are a few things we have to keep in mind:

1. Communicate how suggestions are selected. It is important to communicate very clearly to the users how the final selected suggestions (the winners) will be selected. Otherwise everyone is expecting that the whole process works like an election in which automatically the suggestions with the most votes will be honored, and users will be put off once they find out that this is not the case. The selection process needs to be formulated and presented in a transparent way (e.g. by explaining who is in the selection panel, by what criteria are proposals selected, what are no-go conditions, etc.).
    
2. Avoid cherry-picking. The more the picked winners deviate from actual results of the vote, the less buy-in we will get on those picked suggestions. It might be tempting for governments or managers of organizations to pick suggestions that match most what they want to do anyway, no matter how many votes the suggestions received. However, it cannot be overstated that picking a suggestion as a winner that nobody identifies with will have no positive change management effect. In fact, it is likely to increase the perception that those in charge of formulating the agenda are just cherry-picking what they like for themselves, which creates a sense of disempowerment we need to avoid in any participative process.
    
3. Don’t distort suggestions when clustering. Often we will have to aggregate different suggestions into one in order to capture a cluster of submissions. When doing so we need to be very careful not to distort the original meaning of a suggestion. Otherwise we can face a situation in which e.g. a suggestion that receives a lot of votes is stripped off the very points that made it a popular suggestion in the first place, just so it can fit within a bigger cluster of suggestions. Even when idea is acknowledge, for the person submitting the original popular suggestion, seeing one’s own idea promoted in a distorted manner can be disempowering as well.
    
4. You have to follow up. No excuses.. No matter how great the participation was, and how well the final prioritized agenda reflects what the crowd wanted, if we don’t actually commit to following up on this agenda, we can destroy all the work we’ve done. If we open up a process that promises to take the views of the people we asked to contribute seriously (no matter if it is staff in an organization or the public in a process like the Rio Dialogues), we have to mean it and follow through. Otherwise the group in charge of the agenda loses all legitimacy, as well as the trust it needs to engage its constituency for implementing any changes in the future. The result will be resignation and reluctance to contribute to any similar exercises next time around: We’ve been asked for our opinion in the past, but nothing was done, so why should I care now?

Now, it is understandable that neither governments nor organizations might be able to pick all ideas with the most popular votes as they are because implementing them might have far reaching implications (in terms of cost, feasibility, collateral effects, discrimination of minority groups, etc). There are often good reasons why good ideas cannot be implemented (or at least not right now), but then this needs to be explained to those who were involved in the process. Otherwise people participating in the call for suggestions and the call to vote on them will feel their voice is not heard and feel disenfranchised.

Keep the conversation going

Finally, when presenting the winning proposals, it is important to maintain a follow-up discussion on those results and the process in general, where users can raise their concerns and also make further suggestions on how the proposals can be implemented down the road. Presenting a prioritized agenda based on crowdsourced suggestions is only the starting point of a participative change process. Whatever happens after that should happen with full participation of the crowd as well.