Bridgewater, New Jersey(CNN) Two senior Trump advisers -- one inside the White House and another who recently departed -- signaled Sunday that the knives are out for Steve Bannon, President Donald Trump's controversial chief strategist.
The comments come as a source inside the White House tells CNN that White House chief of staff John Kelly has soured on Bannon, a political operative with deep ties to the 'alt-right' and the former head of the conservative news site Breitbart.
Bannon is seen as pursuing his own agenda, which does not mesh with the power structure Kelly is putting in place, the source added.
National security adviser H.R. McMaster was asked three times by NBC's Chuck Todd on Sunday whether he can work with Bannon in the White House. McMaster dodged the question each time and eventually left it an open question as to whether Bannon is motived by advancing the President's agenda.
"Can you and Steve Bannon still work together," Todd asked.
"I get to work together with a broad range of talented people," McMaster said. "It's a privilege every day to enable the national security team."
Asked again, McMaster said, "I am ready to work with anybody who will help advance the President's agenda and advance the security, prosperity of the American people."
"Do you believe that Steve Bannon does that," Todd asked.
"I believe that everyone in the White House ... should be motivated by that goal," he said.
The more blunt comments came from recently ousted White House communications director Anthony Scaramucci, who told ABC that Trump needs to "move away" from Bannon and Breitbart.
"If the President really wants to execute that legislative agenda that I think is so promising for the American people ... then he has to move away from that sort of Bannon-bart nonsense," Scaramucci said, using a portmanteau of Bannon and Breitbart, the outlet Bannon once said was a "platform for the alt-right."
Scaramucci, who was fired after a vulgar recording of him slamming Bannon was published by The New Yorker, would not say whether the President will fire Bannon, but did say that he spoke with Trump this week.
"I think the President knows what he's going to do with Steve Bannon," Scaramucci said. "It's his decision. But I mean at the end of the day, I think the President has a very good idea of who the leakers are inside the White House. The President has a very good idea of the people that are undermining his agenda that are serving their own interests."
A second source told CNN on Saturday that it's not only Kelly and Bannon whose relationship is on the rocks, it is also Bannon and the President.
Bannon's ouster has been rumored in the past, with tension in the White House spilling out in public on an almost weekly basis.
His job was on the line in April, according to a source close to the White House, after Trump grew frustrated with his inability to cooperate and work with others. The chilling came as Bannon fought with Jared Kushner, a top Trump aide and the President's son-in-law.
"I like Steve, but you have to remember he was not involved in my campaign until very late," Trump told the New York Post in April. "I had already beaten all the senators and all the governors, and I didn't know Steve. I'm my own strategist and it wasn't like I was going to change strategies because I was facing crooked Hillary."
Trump added: "Steve is a good guy, but I told them to straighten it out or I will."
But Kelly's rise to White House chief of staff -- he was given the role last month when former chief of staff Reince Priebus resigned -- has reignited the churn of speculation as Kelly looks at revamping the press and communications team.
One source said rumors of Bannon's demise have been exaggerated in the past, but that there are serious conversations happening now about whether there is a place for him in the administration going forward.