Holding a team on 4th down is like getting a fumble or an interception. Its much more of a boost than giving up three points.
On the other side, an offense that can come away with three points is much more stable than having to deal with getting stuffed on 4th down.