The setting in Home > Options > VaultWiki: Wiki Links regarding opening in a new window only applies to auto-links. If you use an actual WIKI BB-Code tag, you must use |BLANK
Please clarify your report with respect to this information. If a BB-Code is not working as expected, some example code will be helpful.