Could this be your Charles, age 6 in 1850? He's in NJ, any other info would be helpful-
1850 United States Federal Census
Name: Charles Hagaman
Age: 6
Estimated birth year: abt 1844
Birth Place: New Jersey
Gender: Male
Home in 1850 (City,County,State): Howell, Monmouth, New Jersey
Family Number: 115
Household Members:
Name Age
Richard Hagaman 51
Maria Hagaman 48
Mary Hagaman 20
Lena Hagaman 18
Alfred Hagaman 16
Joseph Hagaman 15
Richard Hagaman 13
John Hagaman 8
Charles Hagaman 6
Source Citation: Year: 1850; Census Place: Howell, Monmouth, New Jersey; Roll M432_456; Page: 110B; Image: 230.
Was his wife Lydia? I found a Charles S. Hagaman who was a farmer in Livingston County, Missouri, his wife died in 1916. Where did Charles die? Do you have his death record?
haven't been able to figure that out yet, I'm trying to find him in 1850/186-. but he is on the Civil War records, you might order his Pension record from NARA?
How about this family, do you recognize the siblings for this Charles?
1850 United States Federal Census
Name: Charles Hagerman
Age: 5
Estimated birth year: abt 1845
Birth Place: New York
Gender: Male
Home in 1850 (City,County,State): Brighton, Monroe, New York
Family Number: 326
Household Members:
Name Age
David Hagerman 33
Linona Hagerman 32
Eliza J Hagerman 10
Wallace Hagerman 8
Charles Hagerman 5
Corintha Hagerman 3
Ellen Parker 25
Source Citation: Year: 1850; Census Place: Brighton, Monroe, New York; Roll M432_528; Page: 335B; Image: 398.
I found both David and Livonia Hagaman in Mount Hope Cemetery in Rochester, NY, and her maiden name appears to be Sherman. She died in 1862, age 45. David died in 1897, age 81, so these seem to be strong possibilities for Charles's parents. I haven't been able to find Charles's death record on the Missouri index for some reason, though I did find his wife, Lydia.
I believe this is the same family in 1860, in Monroe County, NY, indexed as Hganor, appears to be Haganor, but I think it is the same family, because of the name and age similarities.