John Anderson was born on 27 January 1855 at Saltcoats, Scotland, the son of John and Agnes (King) Anderson. In 1875 he came to St. John's to work as a draper's assistant with the firm of James Baird. Nine years later he left this firm to form a drygoods partnership, which was dissolved in 1887. In that year Anderson formed the firm of John Anderson popularly known as "The Great Provider" that was a general importing business dealing in drygoods, millinery, boots and shoes, and clothing. He was married twice: first, on 18 November 1884 to Amelia Murray of St. John's and they had two sons, Hugh and John Murray; and second, on 25 December 1917 to Clymena March of Saint John, New Brunswick.

During the early 1900s Anderson was active in the political life of St. John's. In 1900 he won election in St. John's West as a member of Sir Robert Bond's Liberal Party, which received a large majority in the general election held in that year. He did not seek re-election in 1904, but in 1905 accepted an appointment to the Legislative Council. He was also successful in municipal politics, being one of six councillors elected in the 1902 civic election. However, in 1906 he failed in his efforts to win the mayoralty, finishing a distant third to the winner, Michael P. Gibbs, who had strong labour support, and to the incumbent, George Shea. Eight years later he returned to municipal life when he accepted an appointment to the Municipal Commission, which the government nominated to administer the city from 1914 to 1916.

Anderson's name is probably most associated with Daylight Saving Time, which Newfoundland has enjoyed since 1917. In that year the government adopted at Anderson's insistence legislation giving the people each day an extra hour of daylight (by setting their clocks ahead an hour) between the months of June and September. Subsequent legislation changed the period to between the months of April and October. Anderson first had become fascinated with this idea in 1907 and later that year met the London architect, William Willett, who had been promoting it in England. His name was also associated with "Anderson's Houses", a range of 30 houses constructed in 1920 on Merrymeeting Road in St. John's by the Dominion Cooperative Building Association. Anderson was the Managing Director of this company, which he had formed with the help of local business, labour, church, and government to provide needed housing for the working poor and returning war veterans. However, the housing venture experienced financial difficulties and in 1924 the government had the company placed in liquidation.

According to the late Albert Perlin in a 1977 article in the Newfoundland Quarterly, Anderson was a "man not to give in easily" to controversy and possessed "considerable oratorical gifts." He was a man "with a formidable appearance, big physically, with an arresting face whose most prominent features were his piercing eyes and aquiline nose that contributed to an imperious look. " On 8 November 1930 he died at his St. John's residence after a long illness.