Harry Houdini was not a "magician" - at least, not in the sense that many people think of magicians today. He (H. Houdini) was an escape artist. Sure, I'm positive that he knew some basic sleight of hand using cards, coins - maybe even sponge balls, who knows. He was famous for his stunts of physical ability, much like some people think of David Blaine today.
The "Trick That Fooled Houdini" was a very basic ACR, performed by Dai Vernon, employing sleights like the Marlo Tilt, Double Lift, and using a double backer. As someone said earlier, he even openly placed it (Houdini's card) second from the top, and then showed that it has risen to the top.
I believe that this was so effective in fooling the great escapist for two reasons:
1) Double backers were not as widely used and known about as they are today. Due to the nature of the gaff card, it couldn't exactly be publicly advertised - it was most likely advertised in magic shops; which Houdini didn't visit too often, because he was not a "simple magician".
2) Even if he had heard about it (the double backer), Houdini did not oft dabble in the art of card magic, therefore he probably had very little knowledge of the use of such a card. Think of the very first time you saw a double backer, way back when you were young (or at least, I was). You, like myself, probably had NO idea what you could POSSIBLY do with a card that had no face. That is, until you began reading the tricks you could perform with it.
Just my thoughts on the Trick That Fooled Houdini.