1 Duolingo English Test

　　The Duolingo English Test is a measure of English language proficiency. The test has been

　　designed for maximum accessibility; it is delivered via the internet, without a testing center,

　　and is available 24 hours a day, 365 days a year. It has been designed to be efficient. It

　　takes less than one hour to complete the entire process of taking the test (i.e., onboarding, test

　　administration, uploading). It is a computer-adaptive test (CAT), and it uses item types that

　　provide maximal information about English language proficiency. It is designed to be user

　　friendly; the onboarding, user interface, and item formats are easy to interact with.

　　This document provides an overview of the design the Duolingo English Test. It contains a

　　discussion of:

　　? the test’s accessibility, delivery, proctoring and security processes;

　　? the demographic information of the test taking population;

　　? the test’s items, how they were created, and how they are are delivered and scored;

　　? and the statistical characteristics of the Duolingo English Test.

　　The test scores are intended to be interpreted as reflecting test takers’ English language ability

　　and used in a variety of settings, including for university admissions decisions.

　　2 Accessibility

　　Broad accessibility is one of the central motivations for the Duolingo English Test. Tests

　　administered at test centers consume resources which limit accessibility: they require

　　appointments at a physical testing center within certain hours on specific dates (and travel

　　to the test center), and carry considerable registration fees. The maps in Figure 1 show the

　　concentration of test centers in the world (left panel) compared to internet penetration in the

　　world (right panel). The figure shows a stark difference in how much more easily an internet

　　based test can be accessed than a test center* . While the ratio of internet access to test center

　　access is a somewhat limited metric — not every internet user has access to a device that can

　　run the Duolingo English Test, and physical test centers can usually handle dozens of test-takers

　　at once — it is still clear that the potential audience for the Duolingo English Test is orders of

　　magnitude larger than those who could be served currently by more traditional test centers. By

　　delivering assessments on-demand, 24 hours a day, to an estimated 2 billion internet-connected

　　devices anywhere in the world for US$49, we argue that the Duolingo English Test holds the

　　potential to be the most accessible valid and secure language assessment platform in the world.

　　?Central Africa is underserved by both models.

　　? 2019 Duolingo, Inc4

　　Duolingo Research Report

　　Figure 1. Global heatmaps comparing the number of physical test centers (left) per 1 million inhabitants

　　(TOEFL?, IELTS?, and PTE? combined) vs. the percentage of internet users (right).

　　3 Test Administration and Security

　　The Duolingo English Test is administered online, via the internet to test takers. The security of

　　Duolingo English Test scores is ensured through a robust and secure onboarding process, rules

　　that test takers must adhere to during the test administration, and a strict proctoring process.

　　All of these procedures are evaluated after the test has been administered and prior to score

　　reporting.

　　3.1 Test Administration

　　Test takers are required to take the test alone in a quiet environment. The Duolingo English Test

　　can be taken in the Chrome and Opera browsers worldwide. In China, the test can be taken on

　　the the 360 and QQ browsers as well. An internet connection with at least 2 Mbps download

　　speed and 1 Mbps upload speed is recommended for test sessions.

　　3.2 Onboarding

　　Before the test is administered, test takers complete an onboarding process. This process

　　checks that the computer’s microphone and speaker work. It is also at this time that the test

　　takers’ identification information is collected and that test takers are informed of the test’s

　　administration rules.

　　3.3 Administration Rules

　　The list behaviors that are prohibited during an administration of the Duolingo English Test are

　　listed below. In addition to these behavioral rules, there are rules for the test takers’ internet

　　browsers. The browsers are locked down after onboarding, which means that any navigation

　　away from the browser invalidates the test session. Additionally, all browser plugins must be

　　disabled.

　　? Interacting with anyone

　　? Allowing other people in the room

　　? Using headphones or earbuds

　　? Disabling the microphone or camera

　　? 2019 Duolingo, IncDuolingo English Test: Technical Manual

　　5

　　? Looking off screen

　　? Moving out of frame of the camera

　　? Accessing any outside material or devices

　　? Leaving the web browser

　　3.4 Proctoring & Reporting

　　After the test has been completed and uploaded, it undergoes a thorough proctoring review using

　　human proctors with TESOL/applied linguistics expertise, which is supplemented by artificial

　　intelligence to call proctors’ attention to suspicious behavior. This process take no more than

　　48 hours after the test has been uploaded. After the process has been completed, score reports

　　are sent electronically to the test taker and any institutions they elect to share their scores with.

　　Test takers can share their scores with an unlimited number of institutions.

　　4 Test Taker Demographics

　　In this section, test taker demographics are reported. During the onboarding process of each test

　　administration, test takers are asked to report their first language (L1), date of birth, and their

　　gender identity. Their country of residence is logged when they show their proof of identification

　　during the onboarding process. There were 23,460 people who took certified Duolingo English

　　Tests between August 1, 2017 and June 30, 2019.

　　The most frequent L1s of Duolingo English Test test takers include Mandarin, Spanish,

　　Arabic, and Portuguese (see Table 1). Duolingo English Test test takers represent 115 unique

　　L1s and 168 unique countries. The full tables of all test taker L1s and countries of origin can be

　　found in the Appendix.