But if someone knows your rules, the password isn't secure anymore, is it?

If you register at a malicious site, then the admin knows your first two and the last two characters of all your passwords. So then the passwords have actually only the strength of a password with five letters.

Or if someone knows that you are using some leetspeak presentation of the domain name in letter 2 to 7 and only special characters in letter 1,2,8,9 then he can also exclude many possible passwords.