BackgroundThe decline in COVID-19 mRNA vaccine effectiveness (VE) is well established, however the impact of variant-specific immune evasion and waning protection remains unclear. Here, we use whole-genome-sequencing (WGS) to tease apart the contribution of these factors on the decline observed following the introduction of the Delta variant. Further, we evaluate the utility of calendar-period-based variant classification as an alternative to WGS. MethodsWe conducted a test-negative-case-control study among people who received SARS-CoV-2 RT-PCR testing in the Yale New Haven Health System between April 1 and August 24, 2021. Variant classification was performed using WGS and secondarily by calendar-period. We estimated VE as one minus the ratio comparing the odds of infection among vaccinated and unvaccinated people. ResultsOverall, 2,029 cases (RT-PCR positive, sequenced samples) and 343,985 controls (negative RT-PCRs) were included. VE 14-89 days after 2nd dose was significantly higher against WGS-classified Alpha infection (84.4%, 95% confidence interval 75.6-90.0%) than Delta infection (68.9%, CI 58.0-77.1%, p-value 0.013). The odds of WGS-classified Delta infection were significantly higher 90-149 than 14-89 days after 2nd dose (Odds ratio 1.6, CI 1.2-2.3). While estimates of VE against calendar-period-classified infections approximated estimates against WGS-classified infections, calendar-period-based classification was subject to outcome misclassification (35% during Alpha period, 4% during Delta period). ConclusionsThese findings suggest that both waning protection and variant-specific immune evasion contributed to the lower effectiveness. While estimates of VE against calendar-period-classified infections mirrored that against WGS-classified infections, our analysis highlights the need for WGS when variants are co-circulating and misclassification is likely. Summary of main pointsUsing whole genome sequencing, we provide direct evidence of waning vaccine effectiveness and variant-specific immune evasion during the Delta wave. Effectiveness estimates against calendar-period-classified infections approximated estimates against WGS-classified infections, however, calendar-period classification was associated with a variant misclassification.