(This version of the story corrects name of professor to Mudde instead of Muddle, adds Erdogan's first name.)
By Anthony Deutsch and Toby Sterling
AMSTERDAM (Reuters) - EU leaders lined up on Thursday to congratulate Dutch Prime Minister Mark Rutte on beating far-rightist Geert Wilders in the first of a series of European elections this year in which populist insurgent parties are hoping to rock the establishment.
The center-right prime minister had trailed in opinion polls for much of the campaign but emerged the clear victor of Wednesday's election, albeit with fewer seats than before.
Wilders, who campaigned on an anti-immigration platform and wanted to shut mosques and ban the Koran, won a third more seats than at the last election but was thwarted in his bid to become the biggest party.
Rutte, whose win helped boost the euro and European shares, called it an "evening in which the Netherlands, after Brexit, after the American elections, said 'stop' to the wrong kind of populism."
A win for Wilders would have been seen as a boost for French far-right leader Marine Le Pen, running second in opinion polls before a presidential election in April and May, and for populist parties elsewhere that want to curb immigration and weaken or break up the European Union.
The sense of relief among European leaders was palpable.
"The Netherlands are our partners, friends, neighbors. Therefore I was very happy that a high turnout led to a very pro-European result, a clear signal," said German Chancellor Angela Merkel, who will run for re-election in September.
French presidential candidate Emmanuel Macron, expected to face Le Pen in a two-way run-off on May 7, said: "The Netherlands is showing us that a breakthrough for the extreme right is not a foregone conclusion and that progressives are gaining momentum."
The risk premium demanded by investors to hold French government bonds rather than safe-haven German bunds sank in early trade to its lowest level in two weeks, although it later widened again as France released new supply into the market via a debt auction.
GIFT FROM TURKEY
With 99 percent of votes counted, Rutte's VVD Party had won 33 of parliament's 150 seats, down from 41 at the last vote in 2012. Wilders was second with 20, and the CDA and centrist Democrats 66 tied for third with 19 each, data provided by the ANP news agency showed.
Rutte is now virtually guaranteed a third term, leading a government that can be expected to continue tightening immigration policy in the Netherlands, already among the strictest in the EU.
A number of parties including the VVD and the third-placed Christian Democrats (CDA), have already adopted most of Wilders' anti-immigration platform, if not his fiery anti-Islam rhetoric.
With his strong second-place finish, Wilders warned Rutte that he had not seen the last of his Party for Freedom. He added that he wanted to participate in coalition talks, even though mainstream parties have ruled out working with him.
"We were the 3rd largest party of the Netherlands.
Now we are the 2nd largest party. Next time we will be number 1," Wilders said.
Rutte got a last-minute boost from a diplomatic row with Turkey, which allowed him to take a tough line on a majority Muslim country during an election campaign in which immigration and integration have been key issues.
"Rutte profited from moving to the right, but also from Wilders having radicalized a lot over the last years and being invisible in the campaign," said Cas Muddle, associate professor at the University of Georgia, referring to Wilders' decision to forego election debates until the final week.
"On top of that, Turkish President (Tayyip) Erdogan gave (Rutte) a beautiful gift."
Turkey has been locked in a deepening row with the Netherlands after the Dutch barred Turkish ministers from speaking to rallies of overseas Turks.
Turkey's foreign minister said the views of Wilders - who wants to close all mosques and ban the Koran - were shared by rival parties and were pushing Europe towards "wars of religion".
Erdogan on Thursday said that while Rutte may have won the election, he had lost his country's friendship.
FOCUS NEXT ON LE PEN
With 13 parties set to enter a fragmented parliament under the proportional Dutch voting system, it will likely take months for Rutte to negotiate a ruling coalition. He will need at least three other parties to reach a majority.
At 80 percent, turnout was the highest in a decade in an election that was a test of whether the Dutch wanted to end decades of liberalism and choose a nationalist, anti-immigrant path by voting for Wilders and his promise to "de-Islamicise" the Netherlands and quit the EU.
Outgoing French President Francois Hollande called the result a "clear victory against extremism", and European Commission President Jean-Claude Juncker called it "an inspiration for many".
France's Foreign Minister Jean-Marc Ayrault tweeted: "Congratulations to the Dutch for stemming the rise of the far-right."
But Mabel Berezin, professor of sociology at Cornell University in the United States, said defeat for Wilders should not be considered a sign that European populism is waning.
"The real bellwether election will be Marine Le Pen's quest for the French presidency, starting April 23 – that is where the populist action is and that is what we should be focusing upon," she said.
While Rutte overtook Wilders in the closing stages of the campaign, years of austerity pushed down his share of the vote. His junior partner in the outgoing coalition, Labour, suffered its worst ever result, winning just nine seats, down from 38 last time.
Wilders' tally of 20 seats is five more than before, but still well below a 2010 high of 24 seats. Support for the two most pro-EU parties, the progressive D66 and GreenLeft, was way up.
Denk, a party supported by Dutch Turks, looked set to win three seats and become the first ever ethnic minority party in parliament.
(Additional reporting by Stephanie van den Berg, Phil Blenkinsop, Thomas Escritt, Brian Love, Alastair Macdonald, Abhinav Ramnarayan and Madeline Chambers; Editing by Mark Trevelyan)