On November 3, 1864 Terrill married Mary Francis Sharp Osbourne in Windsor, Ontario, Canada. They had eight children before divorcing in 1888 in Daviess County, Kentucky. Terrill remarried twice, though he is not known to have had any more children.
Terrill served in the Confederate forces during the Civil War, as a lieutenant of the 5th Regiment of the Kentucky Cavalry, Companies G and E. Joshua also served in the Buckner Guard with other Boone County residents, including Felix Moses. Two of his brothers, Cpt.George W. and Simeon F. Terrill, a nephew, John W. Berkshire and a cousin, Arthur Terrill also enlisted in the 5th KY Cavalry, along with other Terrills in Kentucky. Joshua's brothers William and Robert Terrill fought for the Union.
Joshua Terrill died on December 10, 1909 in Lamar, Indiana.