The new government says it will scrap a public holiday observed since the 17th Century.