Our product team spent the summer engaging in behind-the-scenes conversations with executives from College Board and ACT, Inc. We left with a lot to report from both camps. This post distills the meetings down for initial consumption. New information and resources will continue to become available about the digital SAT during the next 3–4 months, and Compass will help explain what it all means to families and counselors.
THE SHORT VERSION:
- The 21st century dovetailing of the SAT and ACT is about to diverge.
- SAT’s tact is to change with the times; ACT’s strategy is to stay the course. Both make a case for how students benefit from the paths they are each paving.
- The new terrain looks as transformed as anything the test prep industry has previously navigated. It will require more resources, institutional experience, technical expertise, and specialization than ever before to properly respond to the needs of the market.
- Our product & tech teams continue to feel invigorated about the upcoming changes. The advanced work they’ve completed over the last few years positions Compass very well for the approaching changes.
- Compass will offer an early “PSAT experience” to students this fall, as a way to help them develop comfort with the new format.
- Almost all students in the class of 2024 will finish their testing before the arrival of the digital SAT.
- The class of 2025 will be the first class to be confronted by the PSAT and SAT moving to a digital adaptive format.
College Board will offer an international digital SAT in March 2023.
- Test specs are now available here. The specifications include sample items, but little in the way of practice material.
- College Board’s testing app will be available to download this fall and will include four practice tests.
- Khan Academy will release initial practice exercises this fall.
- Students, high schools, and colleges will not need to learn new scoring. College Board will continue with 200-800 scoring, and no concordance is needed. A 620 on a paper test is meant to equate to a 620 on the digital SAT.
- There is every indication that colleges will superscore across paper and digital SATs. Advanced juniors may test on paper in the fall of 2023 and re-test on the digital form in the spring of 2024.
Differences Between the New Digital SAT and Paper Tests: Overview
DETAILS ON THE DIGITAL ADAPTIVE SAT:
- National dates will remain unchanged for now.
- School day testing will enjoy even greater flexibility: broad testing windows, staggering students throughout the day or across a week or month. Easy make-ups for absences. However, there will be limits to re-testing within a window.
- National testing could eventually offer more flexibility: staggered start times throughout the day, etc.
- Students will test at school (this is not an at-home test) and have many options for testing devices. Students may use their own laptops (Windows or MacOS), iPads, school-owned desktops and laptops, and school-managed Chromebooks.
- Bandwidth requirements are minimal. The entire test is cached and encrypted, so an internet disruption should not prevent the completion of an exam. The exam results can even be uploaded after the test has already been completed. When a proctor says, “Pencils down,” on a paper test, it is difficult to enforce securely. The computer can easily end the student’s exam and keep the results secure.
- There is not yet a firm answer on how fees may be affected.
- Existing accommodations will be maintained and most will be provided in the digital, adaptive format.
- Students with extended time will not be allowed to end a section early to advance.
- For accommodations that cannot be provided digitally (e.g., Braille), students will take a longer, 3-hour, non-digital and non-adaptive exam.
Stage Adaptive, The Basics
- The adaptive nature of the exam is critical to making it a much shorter test.
- When a student connects the app to the internet and starts a test, the app downloads three sets of questions (modules) per section to the testing device.
- Each section is broken into two stages:
- Reading & Writing contains two stages of 27 questions in 32 minutes
- Math contains two stages of 22 questions in 35 minutes
- All students take the test in the following order: Reading & Writing Stage 1, Reading & Writing Stage 2, Break, Math Stage 1, Math Stage 2
- The SAT is not item-adaptive where every question varies based on performance (like the GMAT). Rather, it is stage-adaptive (like the GRE). It will only adapt once for Reading & Writing and only once for Math. This lowers the stakes on any one question and also preserves the ability to go backward and forward within a stage.
- The first stage of each section is not adaptive (item difficulty will not “adapt” to a student’s performance) and it contains a full range of question difficulty.
- At the end of the first stage, the test will choose which of the two remaining sets of questions (modules) is an appropriate level of difficulty: one module will be on average easier and the other module will be on average more difficult. This shift in difficulty is critical to getting to an accurate score on a shorter test.
Stage Adaptive, The Technicalities
- A simple right/wrong tally does not make sense on an adaptive test, which means students will never find out how many questions they got right or wrong. Students will not know exactly how their score was determined. A published scale makes little sense when everyone has taken a different exam.
- Students will not be given access to their problems, because College Board will re-use some problems in the future. In lieu of releasing 3 actual tests per year via the Question-and-Answer Service (QAS), additional sample tests will roll out periodically.
- Two questions per stage are unscored, equating questions. They will not be counted toward the calculation that determines which second stage the student receives nor will they be counted toward the overall score. College Board includes these questions to gather student performance data so that they can be included in future tests. Previously, this was handled by a fifth section that was easily identifiable as an “experimental” section. Students will not know which questions are unscored.
- Still to be determined: College Board is currently engaging in pilot studies in part to determine the best scoring method. One possibility is that they will use rights-based scoring, the simple tally of right and wrong, to determine which second stage module to deliver to a student. For example, they may say that the cutoff to advance to the harder second stage is 12 out of 20 correct. Another possibility is that they will use a form of scoring called Item Response Theory. Questions can have different weights under IRT. They can even have different weights based on score level. IRT can produce an optimized picture of which problems and weightings best place a student into a particular score range, but it is a more complicated model.
Test Security & Technology
- Students testing together will encounter differing versions of the test, even on the first stages. The test draws from a large pool of problems but the created sets of problems will meet the same content standards and provide an equivalent experience.
- The risk of a test being compromised is greatly reduced—no more locked drawers, sealed booklets, or delayed mail items.
- Students at the same site can be started at different times, since there is a greatly reduced risk of sharing problems within a room or during breaks.
- Students will use the equipment provided by the school or site or bring their own laptop or tablet. They can also incorporate external devices such as a mouse, keyboard, or stylus for personal comfort. College Board will establish lending programs for students who do not have approved devices.
- The test will be administered in a locked-down app; students will not be able to open other applications while testing.
- Students will be expected to arrive with fully charged devices and the testing app pre-installed. Test centers *may* provide power to standard-timed examinees and *must* provide it to those with extended time.
- Scratch paper will be provided because, for example, the app will not allow for the marking-up of math diagrams. A pencil or pen is fine.
- The fact that students end at different times may create disruptions, although College Board maintains that its pilot testing has gone smoothly.
- If a power issue interrupts the test, it can be restarted with a proctor authorization. Deliberately powering down and firing up again is cheating and explicitly forbidden. Cheating tactics that try to exploit the digital form factor are unlikely to work, as College Board effectively has a full transcript of what occurred.
- Students will have their own timer in the app, so the problem of mistiming by proctors or the lack of notifications should be eliminated. The timer can be hidden up until the last 5 minutes of each stage.
- Students can move back and forth among questions, but only within the current stage.
- The app includes built-in features like a highlighter, answer eliminator, and question flag to help students track their progress through questions and identify questions that need further review.
Test Content and Question Format
College Board’s position is that scores on the paper SAT and digital SAT are interchangeable because the underlying content is not changing. However, there are some significant differences.
Reading and Writing
- Instead of two discrete sections, reading and writing have been combined into one section.
- Reading passages are much shorter (25-150 words) with one question per text instead of 11 questions alongside a long passage (600-700 words).
- “Great Global Conversation” as a reading passage type has been removed.
- Questions about graphics—tables, bar graphs, line graphs—have been simplified to avoid the former trickiness.
- Writing questions will no longer test commonly confused words or idiomatic phrases.
- The “No Change” option in writing questions is removed.
Sample Reading Questions
This example demonstrates the dramatically reduced length of SAT reading passages.
Sample Writing Questions
- Imaginary or complex numbers are no longer tested.
- Student-Produced Responses can now be negative (and in that case include an extra digit).
- There are no question sets with common info—every question is now discrete.
Sample Math Questions
RESPONSES AND UPDATES FROM THE ACT:
- ACT wants to continue to be seen as the reliable option, since the new digital SAT will be the third different SAT offered in just the last 8 years.
- However, ACT is also surveying stakeholders on their thoughts regarding a shorter exam.
- Rumors of dropping the Science section can be put to rest.
- The optional Writing section will remain to comply with state contracts and for students testing internationally.
- Section Retesting is off the table for now.
- ACT is working on making the test less speeded.
- Paper exams will remain available even if a digital option is someday provided to all students at national sites. Digital testing at national sites is being considered and they are still working through logistical questions with their site partners.
- They already offer a digital test abroad and in special administrations domestically, but it is just a version of the paper exam for the computer.
- ACT has not ruled out adaptive scoring for a revised digital exam.
- Test site shortage—especially in regions like California—is a known problem with no clear strategic solution. Where ACT has strong traction with state and district testing, it’s less problematic. They have no formula for getting schools to opt back into national testing. Both ACT and College Board are succeeding with school day testing but with the unintended effect of reducing the incentive for those schools to also participate in national testing.