The episode's premise
Disney+'s official press record identifies What If... Captain Carter Were the First Avenger? and images of Peggy Carter, Steve Rogers, Howard Stark and the Hydra Stomper. The series changes familiar events from Marvel films to explore alternatives. Bryan Andrews directed the series and AC Bradley served as head writer.
Source [1]Alternative stories and continuity
Disney's earlier series introduction announced Hayley Atwell voicing Peggy Carter and described the project as an animated exploration of different turns in MCU events, planned for 2021. That establishes an adaptation built around possibilities, not a claim that every previous film has been deleted from continuity.
A character can share a heroic identity across different stories without becoming the same person in each version. Sam Wilson's Captain America film provides another example of why a title, actor and character need to be kept distinct. This introduction avoids disclosing the Captain Carter episode's ending.
Source [2]