According to the book "Family of Freedom: Presidents and African Americans in the White House" (link) by Kenneth Walsh of US News:

But Obama, in his most candid moments, acknowledged that race was still a problem. In May 2010, he told guests at a private White House dinner that race was probably a key component in the rising opposition to his presidency from conservatives, especially right-wing activists in the anti-incumbent "Tea Party" movement that was then surging across the country. Many middle-class and working-class whites felt aggrieved and resentful that the federal government was helping other groups, including bankers, automakers, irresponsible people who had defaulted on their mortgages, and the poor, but wasn't helping them nearly enough, he said.

A guest suggested that when Tea Party activists said they wanted to "take back" their country, their real motivation was to stir up anger and anxiety at having a black president, and Obama didn't dispute the idea. He agreed that there was a "subterranean agenda" in the anti-Obama movement—a racially biased one—that was unfortunate. But he sadly conceded that there was little he could do about it.

.
1. This isn't just an attack on the teaparties but on white people in general, and it's of course reminiscent of Obama's "bitter clinging" remarks from two years earlier. It's not clear whether Obama himself used the phrase "subterranean agenda", or whether he just agreed with someone else who used it. However, expect "subterranean agenda" to become as famous as "bitter clingers". It's also not clear whether Obama wanted his comments to become public. The last sentence is probably an example of a sleazy, passive-aggressive attack.

2. The closest whites have to a defensive group of their own akin to the NAACP are the tea parties. Yet, the only people in that movement who might be smart enough to intellectually challenge Obama and show how he's wrong are only in it for the money. The core "principles" of the teaparties are all about the money, and they explicitly ignore "social issues". So, when they're attacked using a "social issue" like the race card, they don't know what to do, or they help their opponents (see also the section of the main teaparties page).

Other than check-cashers like Dick Armey of FreedomWorks and others who are linked to the Koch family, the rest of the teaparty movement lacks the ability and the sanity to intellectually challenge Obama. The teapartiers are the loudest opposition to Obama and, not only are they the dumbest opposition to Obama, but they try to undercut smarter opponents to Obama (through red-baiting, lying, smearing, etc.) Opposing Obama is a job for smart people, not for useful idiot sign-wavers who play dress-up.