Manipulation gets this classification because it is one of the more pure forms of magic and no, it is not limited to "mime" acts such as you describe. I talk quite a bit during Cups & Balls, my Sponge Balls & coin work when I did it. . .
. . . if you didn't catch that let me be more blunt. . .
we use "manipulation" or what is literally
prestidigitation (magic created by the digits or fingers) in about 90% of the stuff most of us perform, even in Mentalism (billets). So "manipulation" is the foundation of magic and I know of very few buskers that don't rely on it, especially when it comes to those "do something for me now" types.
There was a time not all that long ago (in my youth) when Mentalism was classified as the "pinncale" to a magicians career, the "purest form" of magic that took years to master, which is why all Mind Readers, well into the mid 20th century, usually had greying hair & beards, etc. with many a 20 year old deliberately dying their hair and using make-up so as to look older and thus, more of a legit authority on a given topic. This "mold" was shattered in the 1970s by a small group of up-starts that included the likes of Stephan Minch, Bob Cassidy, Larry Becker and Lee Earle and by the latter 70s included John Riggs, Banachek and Richard Webster . . .
I've gone off topic but only to show how different aspects of magic have been classified as "the top of the heap" but the truth is, we have performers in each area of magic that are true masters that can boggle the mind, defying (apparently) all the methods and logic we thought we knew about the craft. When it comes to the great manipulation acts only two stand out most for me; Shamada and Norm Nielson (especially Norm) when it comes to stage work and of all people Johnny Thompson and the great cardician Martin Nash when it comes to close-up.