Structural variation of plastid genomes (plastomes), particularly large inversions and gene losses, can provide key evidence for the deep phylogeny of plants. In this study, we investigated the structural variation of fern plastomes in a phylogenetic context. A total of 127 plastomes representing all 50 recognized families and 11 orders of ferns were sampled, making it the most comprehensive plastomic analysis of fern lineages to date. The samples included 42 novel plastomes of 15 families with a focus on Hymenophyllales and Gleicheniales. We reconstructed a well-supported phylogeny of all extant fern families, detected significant structural synapomorphies, including 9 large inversions, 7 invert repeat region (IR) boundary shifts, 10 protein-coding gene losses, 7 tRNA gene losses or anticodon changes, and 19 codon indels (insertions or deletions) across the deep phylogeny of ferns, particularly on the backbone nodes. The newly identified inversion V5, together with the newly inferred expansion of the IR boundary R5, can be identified as a synapomorphy of a clade composed of Dipteridaceae, Matoniaceae, Schizaeales, and the core leptosporangiates, while a unique inversion V4, together with an expansion of the IR boundary R4, was verified as a synapomorphy of Gleicheniaceae. This structural evidence is in support of our phylogenetic inference, thus providing key insight into the paraphyly of Gleicheniales. The inversions of V5 and V7 together filled the crucial gap regarding how the "reversed" gene orientation in the IR region characterized by most extant ferns (Schizaeales and the core leptosporangiates) evolved from the inferred ancestral type as retained in Equisetales and Osmundales. The tRNA genes trnR-ACG and trnM-CAU were assumed to be relicts of the early-divergent fern lineages but intact in most Polypodiales, particularly in eupolypods; and the loss of the tRNA genes trnR-CCG, trnV-UAC, and trnR-UCU in fern plastomes was much more prevalent than previously thought. We also identified several codon indels in protein-coding genes within the core leptosporangiates, which may be identified as synapomorphies of specific families or higher ranks. This study provides an empirical case of integrating structural and sequence information of plastomes to resolve deep phylogeny of plants.