For most people, borrowing money now means paying back more money later, so of course it seems reasonable to think the same is true for the government, but as Yglesias points out current interest rates on short term loans to the US government are less than inflation expectations. To put this another way. We can borrow 100 real dollars and pay back 95 real dollars (fake numbers for example purposes). THIS IS FREE MONEY. Under these circumstances, it is insane to believe that we need to borrow less in the short term. Obviously these interest rates wont last forever, and are subject to supply & demand &c, but consider that if we refinanced our whole debt at current rates we would effectively reduce its size.
Everyone (sane) agrees that any austerity measure (whether it be tax increases or spending reductions) will negatively impact the economy, so why do it when investors are willing to give us money for nothing (which we could then use to improve the economy through Keyensian stimulus). In the long run, we have a deficit problem that needs to be addressed, but in the short run people are willing to give us free money that can help address our long term problem by a) reducing our real debt long term, and b) allowing us to accelerate the recovery by increasing demand for goods and services through further stimulus spending.
People are complaining that politicians want to kick the can down the road on deficit reduction. It turns out that that's the fiscally responsible thing to do.