Huh, so I just visited my wiki again and now the duplicates are gone. The only thing I did was adjust the friendly wiki urls (went from index.php?wiki/&title=Prefix:Article to /wiki/Prefix:Article). Though even if I switch back, the duplicates are still gone.
I guess this issue can be disregarded for now. Fingers crossed that it won't pop up again.