Understanding the local atomic order in amorphous thin film coatings and how it relates to macroscopic performance factors, such as mechanical loss, provides an important path towards enabling the accelerated discovery and development of improved coatings. High precision x-ray scattering measurements of thin films of amorphous zirconia-doped tantala (ZrO2−Ta2O5) show systematic changes in intermediate range order (IRO) as a function of postdeposition heat treatment (annealing). Atomic modeling captures and explains these changes, and shows that the material has building blocks of metal-centered polyhedra and the effect of annealing is to alter the connections between the polyhedra. The observed changes in IRO are associated with a shift in the ratio of corner-sharing to edge-sharing polyhedra. These changes correlate with changes in mechanical loss upon annealing, and suggest that the mechanical loss can be reduced by developing a material with a designed ratio of corner-sharing to edge-sharing polyhedra.