If you get in the face of every stealthed person you notice yourself coming across, regardless of consequence that you might be stepping up to the wrong person, than I think you're good.
If you're making a decision based on immediate danger, such as someone telling you they see a blue cloaked person following you, so you snap out at blue cloaked people you spot hidden, then you're probably good.
There's really a lot of room for things to be good or bad to do.
1. If I see someone trying to hide, it means that a coded check has been made and my character was deemed codedly capable of seeing that character in the given circumstances. So I notice them.
2. Chances are that there are plenty of ambient people trying and failing to remain hidden that are also noticed by my character.
3. Given that I can't flip out on everyone who is is trying to go unnoticed, I'll just note them and be weary if I keep running into them or they follow me or they reach for my pockets.
Of course, that is for busy places where crime is fairly rampant. If you are sitting in a small restaurant on green and see someone sneaking about, it's probably a lot less common an occurrence and worth reacting to. Or if I was dipped by someone and they got away but I IDed them and I later see them snooping about, I now consider it fair game to go after them.
The gameplay reality is that there's only so many people sneaking around Red at a time for you to spot, and it'd be lame to constantly pick one guy out of the crowd on the streets, especially if they're your character's enemy. Waiting 'till they've given you more of a reason to suspect they're messing with you specifically (spotting them in an alley, in your character's apartment building, etc.) before reacting is best.
Grey0 has it down.
If your character thinks or knows that someone is after them and is paranoid, I don't think reacting to every possible threat in the environment (by moving along quickly or choosing another route, for example) is unrealistic.
In general, though, if your character is capable of noticing someone sneaking, I think you have every right to react however you'd like. In my mind, sneaking is very obvious. Someone is crawling through bushes, hiding behind trashcans, ninja-cartwheeling through shadows. If you see someone acting like that, it's reasonable to wonder what they're up to, if it's something your character might be interested in.