When not doing "magic" I have frequently worked as a salesman and as such come to loathe those that were "in your face" and pushy. The same goes with magic & magicians; especially the trend of approaching total strangers in their place of business, such as is described here, and showing off . . . first off, you are putting them on the spot in a way that could cost them their job and secondly, doing card tricks in a department store in light years away from a "proper" performance atmosphere. Then again, I feel the same way about those that invade people's conversations at a Mall's food court or someone reading a book while enjoying some coffee at Barns & Nobel, etc. IT'S RUDE!
I'm rather surprised more magicians haven't lost their front teeth as the result of fist collisions but fortunately most of the population will be more courteous and "tolerate" such impositions.
Yes, I'm being quite blunt on this point in that I believe in treating people with a bit of respect and that means keeping what I do in a proper setting. For instance, when I'm at a Starbucks type establishment I might be doing some Card Spreads (tarot) which attract people and encourages them to ask questions . . . in other words I've tossed my lure out into the pond in hope of hooking some fish and reeling them into my world. By approaching people in this way I gain the psychological upper-hand because they have chosen to be a part of what I'm doing vs. me putting them on the spot. . . and no, you don't have to be doing the Psychic stuff, you could be practicing things such as color changes, flourishes, even coin routines . . . you're in your own little world and sucking people in. But here's the kicker. . . be aware of the people around you that start watching. When you see someone that seems really focused, invite them in; do something short and sweet that gets a reaction ESPECIALLY when they have a friend or two sitting with them. Allow their interest and excitement energize the rest of the room; the more people that get involved the more freedom you will EARN.
I've done as much as a full hour under these exact conditions. It allows me to experiment with certain theories and concepts, tune my skills before I go out to do a show, and of course you get to break in new material. The more important thing, at least to my mind, is that it's a Marketing & PR opportunity that allows me to pass out my card or even a brochure should I conveniently have a few handy . . . not that I'd carry such things around with me. . .
On that note however, we must remember that what we do and how we present it will bring focus on us and if we are bad and lack confidence or express too much ego, that Public Relations advantage goes flying out the window and there's nothing you can do to change that, first impressions being what they are.
Yes, there are people that detest magic as a whole and like it or not there are even more people in the world that loathe card tricks. While a solid card worker will get high marks, they are rare. To the general public card tricks are "low brow" schtick frequently relegated to the pub and stag parties. If you happen to live in areas of strong religious influence you will find that playing cards are taboo in that community, something that rarely crosses our minds . . . who's to say that wasn't the case with these co-workers mentioned by the OP?
There are many reasons why you won't get a reaction from people, these are but a few of the reasons not mentioned thus far that we need to think of.