Yeah, that's generally what I do too. I don't like to though. The Blue Crown blog explains my sentiments better that I can:
"If an audience sees you pull out a different deck of cards each time you do a trick, they’re not going to think, “Wow, what a great magician!” They’re going to think, “I...