Histone chaperone Hira, which deposits histone variant H3.3 into chromatin in a DNA replication-independent manner, has been implicated in epigenetic memory and is thought to play a role in both early development and aging. However, there are only sparse data on connections between integrity of embryonic development and healthy aging. The pigmentary system, consisting of differentiated melanocytes and melanocyte stem cells (McSCs) of the adult hair follicle and their precursor melanoblasts in embryos, has been valuable in understanding mechanisms of development, aging and disease. Here, we describe a conditional knockout mouse model, Tyr::Cre Hirafl/fl, in which McSCs, melanocytes and their embryonic melanoblast precursors are specifically deficient for Hira and chromatin deposition of histone H3.3. We find that Hira is required for establishment of normal embryonic melanoblast numbers in vivo, supported by single cell RNA sequencing data, and melanoblast identity in vitro. Despite this, by birth, Tyr::Cre Hirafl/fl mice contain a comparable number of melanocytes as wild type mice, and young adults have normal functioning McSCs and only very mildly hypopigmented hair coat. However, neonate melanoblasts from Tyr::Cre Hirafl/fl mice are sensitive to stress both in vitro and in vivo and exhibit more telomere-associated DNA damage foci, a marker of premature aging, than do those from wild type mice. In line with this, knock out of Hira during embryogenesis in Tyr::Cre Hirafl/fl mice caused a premature defect in adult McSC maintenance and premature hair greying, while inducible knock out of Hira in young adult Tyr::Cre-ERT2 Hirafl/fl mice resulted in no observable defect. These studies of the Hira histone chaperone show that perturbations of in utero embryogenesis can cause only modest phenotypic variations at birth and in young adulthood, but profound abnormalities and features of unhealthy aging in later life.