The Christian Democrats got a quarter of the vote in Baden-Württemberg and Rhineland-Palatinate.
Angela Merkel’s ruling Christian Democrats (CDU) have suffered poor results in two key regional votes seen as an indication of what might happen in September’s general election.
The CDU is on course for about a quarter of the vote in Baden-Württemberg and Rhineland-Palatinate.
The two western states were once conservative strongholds.
Mrs Merkel is due to step down as German chancellor in September, after 16 years in office.
The slump in the CDU vote comes amid anger in Germany at the slow pace of the coronavirus vaccine rollout and a mask procurement scandal.
Several conservative lawmakers have quit over allegations they received huge commissions for arranging government deals to buy face masks.
In Baden-Württemberg the Green Party is predicted to hang on to power with 31.5% of the vote, with the CDU getting 23%, less than at the previous poll in 2016.
In neighbouring Rhineland-Palatinate, the centre-left Social Democrats (SPD) are projected to retain power with 33.5%.
The CDU had led in opinion polls but is predicted to get only 25.5% of the vote.
CDU Secretary-general Paul Ziemiak said it was “not a good election evening for the CDU”.
Both results pave the way for regional alliances between the Greens, the SPD and the liberal Free Democrats and raise the prospect of a similar coalition forming a federal government after the September poll.
The SPD’s candidate for chancellor Olaf Scholz said it showed a government without the CDU or its sister party the CSU could be possible. “A lot is possible,” he said.
Green Party Chairman Robert Habeck said the party would “take this success as a tailwind for the Bundestag election campaign”.
Opinion polls show the CDU’s national popularity slipping from 40% last June, when Germany was being praised for its initial response to the coronavirus pandemic, to about 33% this month.
In January the CDU elected centrist Armin Laschet as its leader but he is not guaranteed to be the party’s candidate for chancellor in September’s election.
His rival for the role is Markus Söder, leader of the CDU’s Bavarian sister party the CSU. The party aims to have the matter settled by late May.
By Damien McGuinness, BBC News in Berlin
The results are even worse than expected, and are being blamed on how the CDU is managing the pandemic.
Rising infection numbers and the slow vaccine rollout have soured the national mood. And there is fury that some conservative MPs have earned huge commissions on government deals to procure masks.
Angela Merkel herself is still popular. But with just six months to go before her final term ends, these results don’t bode well for the chances of her party in the general election.