If you mastered everything in all five volumes, you would be one of the top card magicians in the world. If you just get the first two volumes and master them, you would have all of the fundamentals down.
What do you perform out of Art of Astonishment? There is a tremendous amount of great effects in those books.
My recomendation is to pick up Card College Volumes 1 and 2. They provide the building blocks for the later books.
If you want to get a bunch of older books which are less expensive, get what Anthony recommends plus:
~ Arthur Buckley's Card Control
~ Jerry Mentzler's Counts, Cuts, Moves and Subtleties
~ Hugard's Encyclopedia of Card Tricks
~ Hugard and Braue's Expert Card Technique
~ Scarne on Card Tricks
~ Paul LePaul's The Card Magic of LePaul