First, manually remove the VaultWiki files from your site.
Judging from the error you quoted and the source code of AddOnInstaller, your installer was unable to locate the addon-xml file (VaultWiki doesn't even have one), and attempted to remove the files it had uploaded automatically. There was probably an error with file permissions that prevented it from removing the files. This is something to discuss with the writer of AddOnInstaller (what happens if $this->deleteAll is unable to remove the directories).
As for your homepage being unaccessible, I'm not sure why this might be. Since you didn't actually install VaultWiki (even partially, as AddOnInstaller's source suggests), none of its data or plugins should be present. Even so, go to your XenForo Admin Control Panel, and under Development > Code Event Listeners, find and delete any rogue listeners.
You might also try disabling AddOnInstaller to see if that helps. If you aren't getting an error message, it could also be a file permissions error. Try resetting the group and owner of your XenForo installation as AddOnInstaller may have altered these.
In a failed attempt to delete VaultWiki, AddOnInstaller may have accidentally removed XenForo files. You can try re-uploading XenForo to your site. If this helps, report the error to the coder of AddOnInstaller, that it might delete or alter XenForo files if an addon-install fails.
Once your site is usable again, don't try to reinstall VaultWiki using AddOnInstaller. VaultWiki has its own install system that MUST be used instead. VaultWiki doesn't have XML files that XenForo understands.