A lot of the founding fathers didn't want to include a bill of rights in the constitution because they knew that as soon as you started spelling out some rights, people would start arguing that rights not spelled out don't exist. Which is of course why we get these ridiculous mental contortions about what's covered by privacy considerations, or why something is artistic speech and thus protected.
The counter-argument was basically "don't be silly this is clearly a document about a government of limited and enumerated powers."
Oops.