Elusive Joneses

I have been researching for a couple of years and have discovered a lot.  But now, like many of you, I have hit a wall.  Any help would be appreciated.

Looking for Robert McClure Jones (1823?) and Mary Ann Jones (1825?).  I have thorough records of their child, Andrew Jackson Jones born 1846 in Carter County, Kentucky.  I know his parent's names from two of his marriage bonds.  The problem is that in 1850 at age 4 he is living with a William Jones (b. 1793 Virginia), a Nancy Jones (b. 1827 Kentucky), and a mid-wife named Mary Waddle (b. 1793 Virginia).  In 1860, I believe he is still living with Mary Waddle at age 14.  He signed up in 1863 to the Union army and I have thorough records from that point on.  My question is:  Where were his parents in 1850 and 1860?  I can find no record of them in Carter or the surrounding counties.

Robert Jones