(CNN) President Donald Trump may have made a short-term deal this week to fund the government and raise the debt ceiling, but he's also placed House Speaker Paul Ryan in the crosshairs of the far-right members within his conference once again.
In a week that began with Trump punting the plight of 800,000 young immigrants to Congress and will end with a vote on major spending bill that Trump negotiated with House Democratic Leader Nancy Pelosi and Senate Minority Leader Chuck Schumer, Ryan's job only seems to be getting harder.
"We should have stayed here and put together a plan clear back in July on the debt ceiling," Rep. Jim Jordan, a Republican from Ohio and a member of the House Freedom Caucus, lamented Thursday about leadership's decision to go home for recess. "Lack of preparation sometimes puts you in a position where the choices aren't great. We should have prepared sooner that's why we said stay here not take the longest August recess in we've taken in 12 years in a non-election year."
Virginia Rep. Dave Brat, a member of the House Freedom Caucus, said on MSNBC Thursday that "everybody likes Speaker Ryan. It's just, we've got to aim at a target and hit it."
"Paul, to his credit, is always a nice guy and a gentleman to everybody, right?" Brat said. "But we need to lead and like this vote coming up, it's going to be won with a majority of Democratic votes and 100 to 150 Republicans are going to vote against the big bill tomorrow."
A senior congressional source maintained GOP leaders went united in pushing for longer extension of the debt ceiling increase and had the treasury secretary on their side but in surprising meeting, Trump sided with Democrats.
Despite the fact it was Trump who cut a deal with Democrats, not Ryan, conservatives seem on edge with a speaker who some argue isn't cutting the conservative deals they hoped to get under a Republican President.
On Thursday, the Republican Study Committee, a powerful conservative group in the House, announced its leadership would not support the debt ceiling and spending bill expected to come to the floor Friday.
RSC Chairman Mark Walker questioned whether the package should even get a vote.
"Should it even come to the floor? That's a leadership decision, not mine," Walker told reporters about the package to increase the debt limit.
Walker also suggested Ryan could have put up more of a fight.
"He didn't lead on it but at some point you either push back or you just kind of roll with it," Walker said about Ryan, adding, "I think we as a conference as well as our leadership could do more in pushing back sometimes to stay consistent with what we promised the American people."
Pressed if Ryan should have fought the President more on the deal, Walker admitted he wasn't in the meeting and wasn't sure how much the speaker was "blindsided" by what happened.
RELATED: Republican leaders in the room 'shell-shocked' following Trump deal with Democrats
On Wednesday evening, members of the House Freedom Caucus met with the speaker to outline their priorities. A source with knowledge of the meeting with the speaker confirmed it was a frank discussion of policy objectives. But, publicly, Republican members have tried to downplay any reports that Ryan's speakership is in serious limbo.
Alabama GOP Rep. Bradley Byrne dismissed the reports and mocked the notion that some GOP critics were already floating replacement ideas.
"It's not really happening and some of the names that are out there probably couldn't get 50 votes," Byrne told CNN.
Byrne noted he probably will vote against the debt and spending bill package, but asked about the speaker's history of maneuvering the internal GOP divisions, said, "he handles it very well with real grace and composure, and he's as strong today in the House of Representatives as he's ever been."
But the spending bill was only the most recent hiccup for Ryan this week.
Conservatives are also watching Ryan carefully now as the speaker forges ahead on the always-fraught issue of immigration. In 2013, immigration tore the House apart and only revealed deeper schisms between the base and House leadership. With the future of DACA recipients now uncertain after Trump's decision Tuesday to end the program in six months, Ryan told his conference Tuesday that he plans to try and bring an immigration package to the floor that includes protection for DACA recipients as well as border security.
RELATED: DACA decision appears to shift to Congress, but action tough
But immigration has proven to be kryptonite for Republican speakers in the past.
"Under these circumstances, we will see a speaker who I'll say is significantly more sympathetic than I am to the subject of this matter and less sympathetic to the rule of law," said immigration hardliner and Iowa Republican Rep. Steve King. "If this DACA thing comes forward ... you're gonna see a divided Republican conference."
Allies of the speaker, however, say that Ryan's job is to govern not to cower in fear to those in the furthest flanks of his conference.
"I've never know Paul Ryan to back away from a tough challenge. That's just not who he is. He came here to solve problems, not score political points. He's done that his entire career," said Rep. Tom Cole, a Republican and ally of Ryan's. "At the end of the day, Paul Ryan will do what he thinks the right thing is and I think he'll have enough people who are willing to follow him."
This story has been updated.