You Be the Judge: How Taobao Crowdsourced Its Courts

Editor’s note: In the early days of China’s e-commerce boom, online platforms were awash in counterfeit goods, fake listings, and unscrupulous sellers. But by the late 2000s, a handful of platforms like Taobao had emerged from that chaos to lay the foundations of the world’s largest e-commerce industry.

How did that transformation take place? And what lessons does it offer us today? Those are the questions at the heart of Lizhi Liu’s new book, “From Click to Boom: The Political Economy of E-Commerce in China,” out last November from Princeton University Press. Liu, an assistant professor at Georgetown University and expert in Chinese political economy, examines not just the role institutions played in making China the world’s largest e-commerce market, but also the ways China’s e-commerce boom re-shaped the country.

In the following excerpt, Liu takes a closer look at how Taobao, China’s largest e-commerce platform, experimented with crowdsourced courts and democratic-style governance as a way to manage relationships between buyers and sellers.

When disputes occur, Taobao encourages buyers and sellers to negotiate with each other first. If the feuding parties cannot reach an agreement and do not want to go to court, they can use one of Taobao’s two judicial channels: asking a Taobao employee to adjudicate or using an online jury panel to arbitrate. This section discusses the second channel, a unique Chinese institutional innovation.

Alibaba’s Public Jury was established in 2012 to crowdsource justice. It uses a Western-style jury-voting mechanism to solve online disputes and controversial issues. These jurors are termed “public assessors” by Taobao. Interestingly, the name “public assessor” was drawn from the Chinese talent show “Super Girl” (similar to “American Idol”), which, after the authority shut down its mass voting system, transitioned to using a small panel of audience representatives (or “public assessors”) to vote for the show’s winner. The public jury was widely used by the main Taobao site by 2020 and is now frequently used by Xianyu, Taobao’s used-goods market.

Why did Taobao introduce the jury system? Certainly, as Taobao expanded, the volume of online disputes surged, posing challenges for the platform to handle all disputes by itself. However, according to a former platform employee responsible for designing this institution, the primary motivation was not the caseload. Instead, it was propelled by the complexity of online disputes that proved challenging for the platform to resolve alone. Consequently, they opted to involve users in adjudicating these cases to ensure a fairer process rather than solely relying on platform intervention.

To form a jury, Taobao randomly chooses each panel of 13 jurors from 4 million volunteer candidates; each juror may participate in up to 40 cases per day. The candidate needs to be an experienced Taobao user (i.e., those who have registered for more than a year) with a good online reputation (i.e., those who have a sufficiently high credit rating, as discussed below). This requirement is high enough to prevent most dishonest traders from manipulating votes, but low enough to be inclusive and keep the juror pool large. These jurors are unpaid yet motivated to participate. They gain experience points that can translate into different virtual titles or that can be donated to charity by Taobao as real money.

The jurors review evidence presented by the feuding parties, including product photos, parcel delivery receipts, and chat histories between the seller and buyer. All participants are anonymized in the trial to reduce bias. To ensure justice, no communication is permitted between disputing parties and the jurors, or among jurors. The jurors cast votes within 48 hours and can provide written comments on the trial. The party that secures a simple majority vote wins the case.

Up to date, 1.72 million jurors have resolved 16 million case trials, and collectively cast over 100 million votes. Most cases consist of buyer-seller disputes regarding contract violations, such as complaints that the items received do not match the store descriptions. Other cases involve platform-seller disputes in which sellers believe Taobao has unfairly penalized them for violating certain rules. These jury trials not only ease the judicial burden for Taobao; they also help refine the platform’s rules. In 2013 alone, feedback from jurors led to adjustments of more than 140 policies governing users.

In recent years, Taobao has even experimented with panels of 800 to 1,000 jurors to address complicated issues in market governance. For example, it used to host two brands of baby bottles with similar names: “Doctor Betta” and “Betta.” The former is an internationally renowned Japanese brand, while the latter is an inferior Chinese copycat that has been legally registered in China. Taobao faced a dilemma: keeping both brands could mislead customers, but removing a legal brand might hurt its reputation as a fair market. It thus opened the case to jury trial. The jurors voted to remove the Chinese copycat from the market.

Importantly, jury decisions are enforceable. Taobao can freeze the payment in a dispute, take money from the store deposit (for sellers only), lower the ratings of the users involved, or deny the losing party’s privilege to use the platform. If a party is unsatisfied with the jury decision, it can request that Taobao employees intervene and reexamine the case.

Credit scoring and democratic rulemaking

Two supporting institutions contribute to making Taobao’s private institutions work. The first is Taobao’s credit-scoring service, Sesame Credit, provided by Ant Financial, which is affiliated with the Alibaba group. This credit-scoring service evaluates users’ trustworthiness and creditworthiness and screens out those deemed to be dishonest or untrustworthy.

China has a much lower rate of credit card usage than developed markets and thus lacks the associated credit reports. By late 2013, fewer than 25% of the country’s population had credit records with China’s central bank, while the United States had 85% coverage at the time. Sesame Credit thus intends to leverage online big data to reach the population traditionally underserved by state-owned banks.

Sesame Credit, established in 2015, was China’s first credit-scoring system. It initially used transaction data from 300 million registered shoppers and 37 million vendors who were trading on Alibaba-owned platforms (e.g., Taobao, Tmall, and Alibaba). The system assigns each user or business a credit score ranging from 350 to 950. The idea, in the U.S. context, parallels an individual’s FICO score determined by his or her eBay rating or Amazon feedback. Private credit-scoring services are for commercial use only in China; they are not currently related to the government’s social credit score.

Sesame Credit plays an important role in assisting and reinforcing Taobao’s online institutions. It gives users a strong incentive to maintain a positive reputation by honoring online institutions. A higher credit score renders competitive advantages on Taobao, such as larger microloans from Alibaba. It also brings benefits outside the platform, such as waiving deposits when booking hotels or renting bicycles at businesses partnered with Alipay. These additional incentives to honor contracts on the platform have paved the way to microfinance for sellers. Sesame Credit also protects Taobao’s voting systems from acts of manipulation and fraud, such as the creation of fake user identities and voting bots. Only users with a high score can become jurors and adjudicate user disputes, or voters in the House of Representatives for Taobao Rules.

The House of Representatives for Taobao Rules is the other supporting institution launched in 2015. It gives Taobao users a voice in rulemaking by holding hearings, collecting user feedback before enacting rules, and even opening up certain nonessential platform rules to voting. This institution helps platform rules adapt to users’ changing demands. According to an interview with a Taobao employee, the institution was established to preempt seller revolt associated with rule changes, a recurrent problem for Taobao. This institutional innovation has proven effective, and the employee explained that “no large-scale user protests have taken place since then.”

When Taobao proposes a rule change or a new rule in the House of Representatives, all buyers and sellers with a decent Sesame Credit score can vote and express their opinions on the rule. In addition to ordinary representatives, Taobao also invites professionals and scholars who specialize in the rule-related issue area to evaluate the issue and inform ordinary voters. For a certain period after each vote, Taobao makes the results public and adjusts the rule accordingly. From June 2015 to February 2017, the House of Representatives voted on 42 rules. Each case involved over 10,000 votes.

For example, on Sept. 9, 2015, Taobao proposed raising the buyer compensation that sellers would pay if they missed the agreed shipping deadline. On the voting page, Taobao outlined the key points and the reasons for the rule change. Users had a week to post their comments and vote (Oct. 16–22). On Oct. 28, Taobao announced the results: the majority of users supported the new seller compensation scheme, and the new rule passed. In the same announcement, Taobao also acknowledged some concerns raised by the minority who voted against the change, such as the idea that an increase in buyer compensation would only increase buyers’ incentives to abuse the right. In response, Taobao listed the existing constraints on buyers to prevent this from occurring.

Interestingly, Taobao’s House of Representatives shares characteristics with the mixed government that Aristotle and Polybius (as well as Machiavelli, Harrington, and Montesquieu) thought necessary for a republic’s long-term stability. This structure is sometimes identified as “the one, the few, and the many” — that is, an executive, such as a king; the nobility (sometimes mixed with ecclesiastic lords); and the commons (for example, the non-nobility with a certain level of wealth). Under mixed government, the nobles (like Taobao) were often granted the power to make proposals, and the commons (like the platform’s users) had the power to accept or reject them.

However, Taobao’s House of Representatives is only quasi-democratic. The platform retains exclusive power to choose what rules will be voted upon; only changes to nonessential rules (e.g., those governing user-user rather than user-platform relations) can be proposed. But as a private institution, Taobao’s House made it possible to introduce mass voting into the making of market rules.

Limitations

It is important to note that platform institutions have many limitations. To begin with, there is no perfect institutional design. I had a conversation with a rule designer at Taobao who highlighted an issue in a specific market segment — the sale of seeds. This market faced a downfall because the Alipay escrow system only held the product payment in escrow for up to 15 days. Although the 15-day window is adequate for most transactions where buyers can promptly inspect product quality, it presents a problem for seeds. The authenticity of rose seeds, for example, can only be verified several months later. This resulted in a market flooded with counterfeit seeds, ultimately leading to its collapse.

Second, the enforcement capabilities of digital platforms are inherently limited. Certain counterfeit products can evade detection by online platforms. For instance, I once visited a popular Taobao store selling Yangcheng Lake hairy crabs. These crabs, known for their exceptional quality, command significantly higher prices compared to crabs from other regions. However, a discrepancy caught my attention during my conversation with the store owner. The store owner provided information about the size of their cultivation base in Yangcheng Lake and the yield of hairy crabs per acre. Yet a basic multiplication exercise revealed a glaring mismatch — the total number of crabs they claimed to cultivate was far less than the number they reportedly sold. A local industry expert later told me that the discrepancy is likely due to the sale of both genuine Yangcheng Lake crabs and deceptive “bathing crabs” — crabs sourced from elsewhere that had been briefly immersed in the lake before being sold as genuine Yangcheng Lake crabs. This fraudulent practice is widespread in the region and occurs in both online and offline stores. This instance underscores the limitations of online systems in addressing complex quality issues unless more stringent certification processes are introduced.

Even if platforms can identify violations of their rules, the resulting penalties may not be severe enough to deter future instances of fraud. When a user violates a rule, the typical consequences include a reduction in their rating, freezing of funds in escrow, or, in extreme cases, the suspension of their account. It’s worth noting that while a Taobao account is linked to the user’s national ID, there are ways for the same person to create a new store using someone else’s ID, as acknowledged by a platform employee. Such behaviors are difficult to detect as platforms lack offline enforcement power unless they collaborate with the police. This situation helps to explain why counterfeits and cybercrimes persist in the online marketplace.

Moreover, wherever there are rules, there are people who game them. For example, Taobao’s online rating system has become a victim of its own success. Ratings make sellers value their online reputation so dearly that it has generated a large industry of fake reviews. A platform employee admitted the difficulty of identifying fake reviews: “As we improve our system to identify ‘brushers’ (aka ‘fake reviewers’), the ‘brushers’ also become smarter and try their best to imitate real customers … This is a game of ‘catch me if you can.’”

Interestingly, having more rules does not necessarily improve the quality of institutions. To prevent users from gaming the system (such as by faking reviews), Taobao needs to introduce new rules to fix the loopholes. However, interviews show that frequent rule changes confuse users, and having too many rules leads to inadvertent rule violations. This particularly hurts small sellers or rural sellers who do not follow rule changes as closely as large sellers.

Excerpted from “From Click to Boom: The Political Economy of E-Commerce in China,” published by Princeton University Press. The text has been lightly edited for clarity and is republished here with permission.

Editors: Kilian O’Donnell and Wang Juyi.

(Header and in-text images: Visuals from VCG, reedited by Sixth Tone)