Penalties and interest add up by the day if you haven't paid the IRS what you owe them. And they add up big-time if you haven't filed at all. Every day that you put off taking care of your IRS problem only makes it worse. Did You File and Not Pay? If you did, there's interest being compounded daily on what you owe, which is the quarterly federal short-term tax rate, plus 3%. As of this writing, the IRS is charging 4% per year.