• Register
    • Help

    striker  0 Items
    Currently Supporting
    • Home
    • News
    • Forum
    • Wiki
    • Support
      • Manage Subscriptions
      • FAQ
      • Support For
        • VaultWiki 4.x Series
        • VaultWiki.org Site
    • What's New?
    • Buy Now
    • Manual
    • 
    • Support
    • VaultWiki 4.x Series
    • Bug
    • Special characters like ä are not imported correctly

    1. Welcome to VaultWiki.org, home of the wiki add-on for vBulletin and XenForo!

      VaultWiki allows your existing forum users to collaborate on creating and managing a site's content pages. VaultWiki is a fully-featured and fully-supported wiki solution for vBulletin and XenForo.

      The VaultWiki Team encourages you to join our community of forum administrators and check out VaultWiki for yourself.

    Issue: Special characters like ä are not imported correctly

    • Issue Tools
      • View Changes
    1. issueid=4424 November 8, 2015 1:37 PM
      Alfa1 Alfa1 is offline
      Distinguished Member
      Special characters like ä are not imported correctly

      We have nodes with this character in the name: ä
      This becomes: Ĥ

      & becomes &amp
      etc.

      IIRC I reported this a long time ago.
    Issue Details
    Issue Number 4424
    Issue Type Bug
    Project VaultWiki 4.x Series
    Category Importing
    Status Fixed
    Priority 3 - Loss of Functionality
    Affected Version 4.0.7
    Fixed Version 4.0.8
    Milestone (none)
    Software DependencyvBulletin 3.x
    License TypePaid
    Users able to reproduce bug 0
    Users unable to reproduce bug 0
    Attachments 0
    Assigned Users (none)
    Tags (none)




    1. November 8, 2015 2:11 PM
      Alfa1 Alfa1 is offline
      Distinguished Member
      There are really a lot of such cases.
      Reply Reply
    2. November 8, 2015 2:15 PM
      pegasus pegasus is offline
      VaultWiki Team
      1) In your includes/config.php file, is this line NOT commented?
      Code:
      $config['Mysqli']['charset'] = 'utf8';
      And did you leave the following blank in vault/core/controller/import/config/vw.php:
      Code:
      'charset'		 => ''
      OR
      2) Alternatively, is the line in includes/config.php file commented, or 'latin1'?
      And did you set the vault/core/controller/import/config/vw.php line like so:
      Code:
      'charset'		 => 'utf8'

      If either of these combinations were in effect, that explains why this happened.
      Reply Reply
    3. November 8, 2015 2:27 PM
      Alfa1 Alfa1 is offline
      Distinguished Member
      includes/config.php
      has
      // $config['Mysqli']['charset'] = 'utf8';

      vault/core/controller/import/config/vw.php
      has
      // charset is not defined in LocalSettings.php, but is not guaranteed as 'utf8' in MW
      'charset' => '',
      Reply Reply
    4. November 8, 2015 2:30 PM
      pegasus pegasus is offline
      VaultWiki Team
      Does this bug only affect the names or the body text as well?
      Reply Reply
    5. November 8, 2015 2:33 PM
      Alfa1 Alfa1 is offline
      Distinguished Member
      body text as well.
      Reply Reply
    6. November 8, 2015 2:54 PM
      Alfa1 Alfa1 is offline
      Distinguished Member
      Also in comment signatures.
      Reply Reply
    7. November 9, 2015 1:33 PM
      pegasus pegasus is offline
      VaultWiki Team
      This is something I would like to look into, but it would require FTP access so that I can compare all the different encodings (input to the database, storage in the database, the connection used by vBulletin, the connection used by the importer, and what the importer decides when it tries to detect each of these). It looks like there was a problem somewhere, but unfortunately I will have to go line-by-line to find it, on an import where the problem occurs. Believe me I have done VW3 imports with Asian characters that came out just fine.

      There are a few other issues I've noticed, such as some attachments have invalid description URLs that do not rebuild successfully. Also there was a problem where absolutely no default setting values were applied at installation (and this is the first time I've ever seen that).

      By the URLs you've sent me, I am under the impression that this is was test import. Would it be safe to clear the VW4 wiki data so that I can see what happens as each item gets imported? I understand you must whitelist my IP through a firewall. I'm PMing that info to you now.
      Reply Reply
    8. November 9, 2015 1:53 PM
      Alfa1 Alfa1 is offline
      Distinguished Member
      Yes, this is a test import. I expected that importing this large data set over so many VW version would result in hiccups.
      I will send you the details as soon as another developer is finished with the dev site.
      Reply Reply
    9. December 5, 2015 4:23 PM
      pegasus pegasus is offline
      VaultWiki Team
      Double encoding (e.g. & becomes &amp occurred only for area titles, social group titles, and redirect titles for redirects that are created automatically to deal with high-level conflicts - like two index pages or two areas with the same name. This is now fixed.

      Apparently the other problem wasn't an issue with the importer at all, but rather that vBulletin 3 behaves a little differently from vBulletin 4: it sends the content-type and charset headers before the styles are loaded. Thus, VaultWiki saying that the charset was UTF-8 occurred too late, because a charset=ISO-8859-1 header was already sent. Even though characters were properly converted and the HTML output was fully UTF-8, vBulletin already sent a header to the browser telling it to expect ISO-8859-1 content. This was resolved by moving VaultWiki's code that changes the character set to an earlier location in vBulletin 3.
      Reply Reply
    10. December 27, 2015 11:16 AM
      Alfa1 Alfa1 is offline
      Distinguished Member
      Unfortunately this is not fixed.
      Reply Reply
    11. December 28, 2015 1:15 PM
      pegasus pegasus is offline
      VaultWiki Team
      There were 3 or 4 various bugs here. Which aspect are you referring to?
      Reply Reply
    12. December 28, 2015 1:36 PM
      Alfa1 Alfa1 is offline
      Distinguished Member
      Please see the wiki: H?ufig gestellte Fragen
      This should have been: Häufig gestellte Fragen

      I assume that there are similar issues with special characters.
      Reply Reply
    13. December 30, 2015 9:47 AM
      pegasus pegasus is offline
      VaultWiki Team
      There was a typo in a new and important regular expression (an opening [ was missing). I will reinstall and run the import again.
      Reply Reply
    14. December 30, 2015 3:14 PM
      Alfa1 Alfa1 is offline
      Distinguished Member
      Great. Let me know how it goes.
      Reply Reply
    + Reply

    Assigned Users
    Loading Please Wait
    Tags
    Loading Please Wait
    • Contact Us
    • License Agreement
    • Privacy
    • Terms
    • Top
    All times are GMT -4. The time now is 1:12 AM.
    This site uses cookies to help personalize content, to tailor your experience, and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Learn more… Accept Remind me later
  • striker
    Powered by vBulletin® Version 4.2.5 Beta 2
    Copyright © 2023 vBulletin Solutions Inc. All rights reserved.
    Search Engine Optimisation provided by DragonByte SEO (Pro) - vBulletin Mods & Addons Copyright © 2023 DragonByte Technologies Ltd.
    Copyright © 2008 - 2013 VaultWiki Team, Cracked Egg Studios, LLC.