It's true this could be due to a bad performance or the trick you've chosen or even the time and place you decide to present magic. Or in the worst case: you're persona is not likable.
BUT a lot of people here a dismissing the fact, that somepeople don't just like magic. Saying it's all the performers fault is just a bad attempt to show off, as if it had not happened to you, yeah right.
The secret is to try to recognize what's the cause for this. If the spectator just doesn't care about magic, this could be due to many reasons, it would be best to just thank them for their time. If your performance is not good enought, as in some cases, you need to practice your moves and study the theory of magic.
Take a look at some works of the greats like Arturo Ascanio, Darwin Ortiz, Juan Tamariz, Tommy Wonder... The list could go on and on.
You can also watch performances from all kinds of great magicians from youtube and such other sites. Study how they present their magic and read what they have to say about magic.
There is a reason why there's such a little good magic around these days. Too many people dismiss the study of the theory of magic and the importance of performance. Magic is a difficult performance art, unfortunately the word art gets tossed around a lot, as in my opinion 99% of the magic today, is far far from even resembling art.
If you want to be a great magician you need to put in so much hard work that your magic could be compared to playing the most complicated symphonies with a piano, while the same time reciting Shakespeare, dramatically.
Oh yes, the last thing. PRACTICE your tricks so that you know them inside and out. If you have to think about the modus operandi or even 'what's next', you will lose all the ENERGY in your performance, and that reflects to the spectators interest level. Your focus should be completely on the situation, DON'T JUST FOCUS ON THE TRICK, BUT IN THE SITUATION ITSELF, so you can adapt your lines, tone, pacing, blocking and all the other theatrics, to best fit the specific situation. That way you get the most mileage out of your material and assure the best experience for the spectators. AND don't be afraid of interaction, because performing is all about the interaction between you and the audience, on conscious and subconscious level.
paulpaul