It may just be because they haven't seen you perform yet. I did this with my friend before. He'd be performing street magic, and I'd be acting like a spectator, getting all amazed (which wasn't hard to do, considering I was), and slowly, people stopped to watch for a few minutes before going on...