Since 2011, Klein had lived in a continuing care home in Calgary suffering from Chronic Obstructive Pulmonary Disorder and frontotemporal dementia, a progressive disease that limited his speech and mobility.
Last week it was reported that his condition had worsened and was put under palliative care.
Alberta Health Services announced Klein's death, on behalf of his family, stating that he was surrounded by loved ones and close friends.
Ralph's widow, Colleen Klein, thanked the public for their support.
"The nature of his illness made it very difficult to express his thoughts these past years which I know was a real challenge for him, but Ralph very much knew and appreciated the well wishes and warm messages he received," she said in the AHS statement.
"I want to thank everyone for their support and especially of the caregivers who helped us throughout. It has all made a tremendous difference."
Tributes, on this Good Friday holiday, poured in from around the country. Alberta Premier Alison Redford called Klein a staunch defender of Alberta.
"I am truly saddened by our loss of Ralph Klein," she said in a statement.
"Ralph Klein’s ability to connect with Albertans from all walks of life was absolutely remarkable. He could walk from the Petroleum Club in downtown Calgary to the curling rink in St. Paul and carry on a conversation with absolutely everyone he met. Ralph was a real man of the people."
Prime Minister Stephen Harper said "Alberta and Canada have lost a unique and significant leader."
"While Ralph’s beliefs about the role of government and fiscal responsibility were once considered radical, it is perhaps his greatest legacy that these ideas are now widely embraced across the political spectrum," Harper said in a PMO statement.
And Calgary Mayor Naheed Nenshi said Klein's legacy surrounds Calgarians.
"As mayor, he had a tremendous impact on Calgary as a city government, helping shape both our policy and our culture," Nenshi said.
"Ralph Klein taught us, as Calgarians, that we don’t need to put on airs. We don’t need to pretend we’re something we are not in order to be a truly great city in this world."
Klein entered politics after a stint in the air force and a career in PR and as a journalist.
As a political neophyte, he surprised many in 1980, including himself when he won the mayoral race in Calgary.
According to CBC News, the country got their first taste of him in 1982 when he complained that the "bums and creeps" who moved to Calgary from Eastern Canada were putting a strain on City resources.
Despite his rough around the edges demeanor, Calgarians rewarded the man with reelection in 1983 and 1986. According to his legislative bio, Klein’s major accomplishments, as mayor, "included the 1988 Olympic Winter Games, Calgary's Light Rail Transit System and protection of the Bow River."
Klein seamlessly made the transition to provincial politics in 1989 getting elected as a member of the legislative assembly for Calgary Elbow. Within three years he took over the party leadership replacing Don Getty upon his retirement in 1992.
And so began the reign of "King Ralph."
As premier he was a staunch fiscal and social conservative which made him a polarizing figure. He faced intense criticism for cutting funding for the arts, for social programs and for healthcare but, in doing so, eliminated the province's deficit, retired its debt and inspired the Jean Chrétien Liberals to strive for a balancing of the nation's books.
He was also credited for stewarding the province through the Mad Cow disease epidemic in the early part of the last decade.
He wasn't perfect — he was a man who had his foibles and didn't try to hide them.
As explained by the Globe and Mail, he "famously threw money at a homeless person after grilling him on why he doesn't have a job; who drank heavily; and who frequently clashed with Ottawa over a series of issues affecting his province, including a stand against same-sex marriage."
Klein retired from politics in 2006.
His last major public appearance was in June 2011 when he attended the opening of a Calgary park named after him.
Last Fall, he received the Order of Canada at a short ceremony at Calgary City Hall. His wife, Colleen Klein, accepted the honour on his behalf.
What made Klein so popular?
As chronicled by CBC News, Maclean's magazine brilliantly articulated the appeal of Klein in a column in 2004:
"How to explain the Klein phenomenon? Part of it -- a big part -- comes down to personality. Klein is someone a lot of Albertans have trouble disliking, even when they disagree with his policies... Despite more than two decades in public life (he also spent three terms as the wildly popular mayor of Calgary), Klein retains the common touch.
He lives much of the year with his wife, Colleen, in a modest, three-bedroom bungalow in his home city, drives a 1977 Volkswagen Beetle and favours blue jeans and casual shirts when off duty."
(Photo courtesy Canadian Press)
Are you a politics junkie?
Follow @politicalpoints on Twitter!