The American wealthy are hands down the most philanthropic in the world. Americans dominate global giving lists and surveys consistently show that the U.S. rich are far more likely to make philanthropy a priority.

But some say they could give more. One recent study found that a large share of people making $200,000 or more give only 2.8 percent of their income to charity. Other studies show that multi-millionaires donate only about one percent of their wealth to charity (though billionaires tend to give a higher percentage).

Read more: Stars of 'Nerd Philanthropy' Dominate Top Giver List

Warren Buffett and Bill Gates launched their Giving Pledge in large part to persuade the super-rich and the non-super-rich alike to give more to charity. As Buffett told me in 2011, "The hope is that our larger population ends up giving a larger proportion of their income to fund philanthropy."

So why don't the rich give more?

A new study of multi-millionaires offers some answers. The study, by SEI Private Wealth Management, found that 82 percent of wealthy families believe that having more money means you have a greater obligation to be philanthropic.

But the respondents (worth $10 million or more) listed three main reasons for not giving more. First, nearly half said they needed more confidence "that the level of their wealth would continue to support their lifestyle and their family." Second, they said they would give more if the markets improved.

Finally, a third of those polled said they needed to "find something they could be more passionate about."

Read more: Did 2013 Tax Hikes Slow 2012 Charitable Giving?

The first two reasons aren't all that surprising. If they had more money, the wealthy would give more of it away.

But the third reason is worth noting. While more money helps, it's also important to be motivated by a cause. And if you care enough about a particular problem, you'll be more inclined to sacrifice some of life's comforts to solve it.

Money, in other words, isn't the only driver of philanthropy. It's about the heart as much as the wallet.