Well the reason is, that you seem to have missed that the string gadget for entering the search string accepts AmigaDOS wildcards. Therefore, the [ is a special reserved character and have to be escaped to match the literal character itself. So if you want to search for a [ character with an AmigaDOS pattern aware method you have to use:
'[SPAM']
where the ' character escapes the reserved state of the [ and ] chars. However, even more precisly would be to use:
'[SPAM']#?
instead and turn off the "Substring" option. That would then match only subjects that actually start with [SPAM] as there might even be non-SPAM messages contain that string somewhere.
NOTE on searching for [SPAM]:
Matching SPAM messages on the literal [SPAM] marker in a subject line is more or less a bad idea as it would cause (even if correctly escaped) many non-SPAM messages to get filtered incorrectly. Normally SPAM filter add an additional header line like 'X-Spam-Status:' to each checked email. So to reduce the amount of incorrectly matched mails you would be better off searching for that optional header line and search if it starts with "Yes".
The Online-FAQ of www.yam.ch