I could tell you that situation was an anomaly - that it will never happen again. But honestly, that wouldn't be true. The reality is that situations like that are part of the action, and the nature of the beast. With magic, and entertainment by any means, there will always be a certain subsect of people that aren't interested, aren't enthusiastic, or otherwise just aren't a good audience.
It happens, and it will happen again. Whether you're John Smith or David Copperfield, even the best performance will result in some people NOT liking it. Simply put, you can't please everybody. Even the best Hollywood movies - with flawless critical reviews and 98% on Rotten Tomatoes - will result in some people leaving the movie theater saying "that sucked - I hated it!"
But here's the good news: that's okay! It's okay not to please everybody. It's okay that some people may not like your performance, your routine, or even your whole show. It's okay that some of your performances get better reactions than others. Your goal as a performer, and as a human being in general, is to please as many people as possible - and make people happy. But the reality is that goal - 100% success rate - is unattainable. David Copperfield makes a car appear on stage five feet away from the audience's face, and while 99.9% of the audience is SHOCKED in complete astonishment, there are a small minority of people, every night, that don't react at all.
Why? It could be a number of factors. With yourself, with Copperfield, or with any great movie, many things can affect how we respond to different situations. If I'm in a bad mood, or I'm exhausted, my response to a magic trick isn't going to be that spectacular, either. If I'm stressed out, or I'm thinking about something else, I'm likely predisposed NOT to be a good audience of any sort at that particular moment. Instead of entering the performance as a neutral observer, I would be entering the performance already at a negative, like -15% happiness level - so the performer would have to move the needle 15% just to get me to zero.
That is not to say that we, as performers, can't do something about it. After any performance, it's good to reflect and think about what you could have, perhaps, done differently or better. Copperfield watches every single show on his iPad when testing new illusions - we can always improve. We can always do better. In this case, what could you have done differently that may have affected the outcome? Perhaps nothing, but the answer may also be involving all of the audience more, so those two "tough spectators" were engaged and more on your side. It could be learning and thinking about what they did see; what may have tipped the method to them? What could you as a performer do to prevent that next time?
All of these things are food for thought, and good things to keep in mind. But NONE of these things should stop you from performing more, and more often. Use this experience as fuel to inspire you to do better - to learn from this situation and turn a negative into a positive. In the process, with each performance, you'll find yourself getting better and better. Your confidence level will rise. You will speak more eloquently. You will no longer be overcome with nerves. And most importantly, you'll begin having more and more fun. In the end, that's what it's all about. If you're having fun, your audience will too.