Washington(CNN) Justice Anthony Kennedy, the man who so often determines the outcome of the most controversial Supreme Court cases, is himself the center of brewing speculation.
Will he stay or will he go?
The rumors have swirled for months and the 80-year-old justice has done nothing either personally or though intermediaries to set the record straight on whether he will step down.
Helping drive the speculation, dozens of Kennedy's former law clerks traveled to Washington this weekend to participate in a private clerk reunion that occurs regularly -- and many of them wondered if it will be their last chance to meet with him while he is still on the bench.
At the end of a dinner with the former clerks Saturday night, Kennedy addressed the crowd, saying he had heard some speculation about an announcement tonight, "and here it is," he said: The "bar will be open after dinner."
But sources close to Kennedy say that he is seriously considering retirement, although they are unclear if it could occur as early as this term.
His departure would cause a seismic shift and offer President Donald Trump a chance to continue reshaping the court. Trump's first nominee -- Justice Neil Gorsuch, himself a former Kennedy clerk -- joined the court earlier this year.
Like no other justice in recent history, Kennedy has cast the vital swing vote in cases that grab the countries' attention.
To liberals he is a hero for Obergefell v. Hodges -- a landmark opinion that cleared the way for same-sex marriage in 2015 and will likely be his most lasting legacy.
"They ask for equal dignity in the eyes of the law and the Constitution grants them that right," Kennedy wrote.
To the delight of abortion rights supporters, Kennedy voted to reaffirm the core holding of Roe v. Wade in 1992.
"As the court's most important Justice -- at the center of the institution's ideological balance -- Justice Kennedy's ability to bridge the divide between left and right on critical issues such as the right to access abortion cannot be overstated," said Elizabeth Wydra, president of the Constitutional Accountability Center. "Replacing Justice Kennedy with a Trump nominee would almost certainly sound the death knell for Roe, just as candidate Trump promised during the 2016 campaign."
But nine years later, he sided once again with the liberals on the court to strike down a Texas law that abortion rights supporters thought was the most strict nationwide. Without Kennedy's vote, the law would have been allowed to go into effect, inspiring other states to pass similar legislation.
In the same term, Kennedy pivoted on the issue of affirmative action when he voted for the first time in favor of a race-conscious admissions plan at a public university.
After that term, former acting Solicitor General Neal Katyal said, "It is very much Justice Kennedy's Court."
"You can't understand how important his affirmative action opinion is without understanding his earlier jurisprudence," said Katyal. "For decades, he has been the court's most eloquent voice on the need to be color blind -- why he changed his mind is something historians will debate for decades."
However, sometimes Kennedy voted with the four conservatives on the bench. It was Kennedy who penned the majority opinion in Citizens United v. FEC -- striking down election spending limits for corporations and unions in support of individual candidates.
He's also sided with the right side of the bench on issues such as gun control and voting rights. Kennedy joined Chief Justice John Roberts' 2012 opinion, Shelby County V. Holder striking down a key provision of the Voting Rights Act.
And Kennedy sided with George W. Bush in the case that essentially decided the 2000 presidential election for the GOP candidate.
On one side is his age -- a desire to spend more time with his grandchildren is driving any decision, and in many ways he has already established an enduring legacy on the court.
In terms of a replacement, Kennedy might take comfort in the list of 20 judges Trump has vowed to draw from when considering the next vacancy on the court.
Another consideration is that if Kennedy were to delay his retirement for a year, his replacement would face confirmation during the mid-term election year, something that could further inject politics into an already controversial process.
On the other hand, Kennedy is well aware of his role on the court and could be alarmed by how politicized the confirmation process has become. Indeed, Republicans were forced to change Senate rules to make it easier to confirm Gorsuch after Democrats objected.
Kennedy might think it would make sense to remain on the bench until the political climate simmers down -- although there's no guarantee that would ever happen.
Retiring at 81 would not be all that much different than retiring at 80 and he would get to serve longer with Gorsuch as well as take up a case on next term's docket concerning partisan gerrymandering -- an issue that might once again keep Kennedy in the spotlight.