Mt. Oliver Baptist Church was established on September 22, 1907. Tommy and Alec Oliver gave the land for the church. The church was dedicated on the third Sunday of May in 1909 and also had the first communion. Members of the first church came from Willeo, First Baptist of Roswell, Lebanon and New Home. Total membership was twelve, 5 male and 7 female.
R. E. Greer 1907-1911 was moderator in 1907-1911 W. H. Earley was moderator in 1911-1912 George Compton was moderator in 1912-1914 W. H. Smith was moderator in 1914-1916 Henry Phillips was moderator in 1917-1929 W. H. Earley was moderator in 1929-1930 J. I. Holbrook was moderator in l930-1944