The Daily Telegraph and other newspapers disclosed on Saturday that a member of David Cameron’s inner circle had described Conservative association members as “mad, swivel-eyed loons”.
Speaking during his BBC Newsnight programme, Mr Paxman said that he had been present when high-ranking Tories had used those terms to describe local party members.
Mr Paxman’s comments are sure to increase the pressure on Mr Cameron over the issue, which has caused deep divisions in the Tory Party.
The Prime Minister yesterday attempted to assure party members in an email that neither he nor his inner circle would ever “sneer” at them.
Mr Cameron did not refer explicitly to the “mad, swivel-eyed loons” remark, but insisted that he admired and respected his party’s activists.
Interviewing Tory activist Binita Mehta on his programme last night Mr Paxman said: “What about this swivel-eyed loon[s]…that such language can be used by people near the centre of the party about people who get the party elected?”
He added: “I have heard senior members of your party talking about local activists in these terms.”
The Telegraph has not named the individual responsible for the comments. However, over the weekend Lord Feldman denied he had made the remarks.
Mr Cameron made his peace offer after being accused of “railroading through” same-sex marriage laws following a last-minute deal with Labour to rescue the legislation from Commons defeat.
“I am proud to lead this party. I am proud of what you do,” he said. “I would never have around me those who sneered or thought otherwise. We are a team, from the parish council to the local association to Parliament, and I never forget it.”
An amendment tabled by Tim Loughton, the former Conservative Minister, to widen civil partnerships to heterosexual couples, was defeated by 375 votes to 70, paving the way for gay marriage.
He accused the government and the Labour leadership of doing a "grubby deal" to defeat his measure.
Mr Loughton said that despite being promised a "free vote", MPs were subjected to "a last minute stitch-up between front benches".
Labour sources said Ed Miliband had agreed to encourage his MPs to vote against Mr Loughton's amendment after receiving an "11th hour" appeal for help from the government.
Ministers approached the Opposition to warn that they would not have enough votes to stop Mr Loughton's plan unless Mr Miliband asked his MPs to vote against it, Labour said.
The Labour leader's priority was to ensure that the same-sex marriage Bill passes into law, a source said.
Conservative members and MPs have strongly resisted Mr Cameron’s plan to allow same-sex couples to marry. About 150 Tory MPs, including Cabinet ministers, are expected to vote against the plan in its final reading on Tuesday. Last week, more than half of the party’s backbenchers voted to criticise the Coalition for failing to offer legislation for a European Union referendum.
The disputes over Europe, gay marriage and the “loons” comment have led some Conservatives to claim there is a fundamental difference between Mr Cameron and the party’s grassroots.
Mr Cameron tacitly addressed claims that he surrounds himself with a privileged elite by recalling his own work for the party in his 20s, describing himself as just “one of the volunteers” on a local council campaign.
He wrote: “Since then I have met thousands and thousands of party members. We’ve pounded pavements together, canvassed together and sat in makeshift campaign headquarters together, from village halls to front rooms.”
Mr Cameron also acknowledged his differences with party colleagues, but promised to respect their views. He said: “Does that mean we will agree on everything? Of course not. The Conservative Party has always been a broad church and we must remain so today.”
The UK Independence Party has sought to capitalise on tensions within the party, appealing to unhappy activists to defect, an offer taken up by some councillors.
A Survation opinion poll put the Tories on 24 per cent, two points ahead of Ukip on 22 per cent. Labour scored 35 per cent and the Liberal Democrats 11 per cent.
Mr Cameron is relying on Labour and Lib Dem votes to steer the Marriage (Same Sex Couples) Bill through its final Commons stage. Although he allowed his MPs a free vote, he has personally championed gay marriage and the size of the revolt represents the latest Tory challenge to his leadership.
The main vote on the legislation took place this evening before the Bill moves to the House of Lords, where it is also expected to be the subject of intense debate.
One former Conservative minister claimed that the “aggressive homosexual community” was using gay marriage as a “stepping stone to something even further.”
Downing Street agreed to support a Labour amendment to the legislation to stop Tory rebels “wrecking” the Bill. Tim Loughton, a former education minister, had proposed that civil partnerships, which are only available to homosexual couples, be extended to heterosexuals if gay marriage was legalised.
However, senior government sources initially warned that the amendment, which senior Labour ministers were prepared to back, could have “wrecked” the Bill, as the cost of extending civil partnerships could reach £4billion. It was thought that the Treasury may have been forced to intervene to abandon or delay the legislation if the amendment was passed.
Labour instead introduced its own amendment, calling for an immediate consultation on extending civil partnerships which the Government indicated it was happy to support.
Yvette Cooper, the shadow home secretary, said: “The whole debate is getting sucked into this vortex of Tory infighting, and that’s making it much harder to have a sensible debate about the whole Bill, which we strongly support.”
A source close to Nick Clegg, the Deputy Prime Minister, welcomed the deal. “As Nick Clegg has said, the overriding objective of the Liberal Democrats is to ensure that equal marriage becomes law,” he said.
Conservative backbenchers accused the Prime Minister of restricting debate. Sir Gerald Howarth, the former defence minister, said gay marriage was not in any manifesto and was being “railroaded through the House”.
“I think it is really unacceptable that a measure which is going to have such an impact on our national life, which strikes at the heart of a thousand years of religious and cultural tradition in this country, has been reduced to a couple of hours,” he said.
Tory MPs were also defeated in an attempt to allow public officials to opt out of duties directly related to gay marriage. Those voting for the amendment included Iain Duncan Smith, the Work and Pensions Secretary, Owen Paterson, the Environment Secretary, senior Lib Dems including Simon Hughes, the deputy leader and Tim Farron, the Party President.
A spokesman for Mr Cameron defended the legislation. “The Prime Minister is a big believer in marriage,” he said.
Read the latest Telegraph Politics coverage