Big Board Optimization

[Kaelon]

I see you are moderating the forums.
But all the products are dead still
Nuhit's Nuwiki seems quite primative to vaultwiki.
I don't know why anyone would want to go with nuwiki, even if it was vB4 compatible.

There are some subtle, but significant, differences. For example, NuWiki allows you to use templates board-wide. My understanding is that VaultWiki is designed to only allow template use on a Wiki-forum basis.

[Kaelon]

I believe I've mentioned this elsewhere, but yes I seem to remember a number of big boards that have installed VaultWiki. Of course I have to leave it up to them to chime in with respect to how it performs on their particular setups.

It would be great if there were a testimonials or a list of "example sites", so that those of us who are considering VaultWiki for Enterprise use can be re-assured that, at least this go around, it is ready for prime-time.

[InformationNirvana]

There are some subtle, but significant, differences. For example, NuWiki allows you to use templates board-wide. My understanding is that VaultWiki is designed to only allow template use on a Wiki-forum basis.

Wouldn't the more powerful option be to have templates on a wiki-forum basis ? Is it really that hard to apply all the templates for one wiki-forum ... to all the wiki-forums. And then you also have the power of having some wiki-forum specific templates ?

Seems like a small difference to advocate for a dead product.
Wouldn't it be cheaper for you to convince VaultWiki to make a few changes to make it work like your nuwiki currently ? You want to find a nuwiki developer ... but I think that'll be hard considering most people would be smarter to use vaultWiki.

Are there any other differences ?

[Kaelon]

Wouldn't the more powerful option be to have templates on a wiki-forum basis ? Is it really that hard to apply all the templates for one wiki-forum ... to all the wiki-forums. And then you also have the power of having some wiki-forum specific templates ?

Seems like a small difference to advocate for a dead product.
Wouldn't it be cheaper for you to convince VaultWiki to make a few changes to make it work like your nuwiki currently ? You want to find a nuwiki developer ... but I think that'll be hard considering most people would be smarter to use vaultWiki.

Are there any other differences ?

I don't necessarily "advocate" for a dead product, but there are a few important things to keep in mind. Much of NuWiki's success comes from its open-ended, light-weight design, and the things that are important to my community, and to other Big Board Enterprise customers, may be very different from what a small hobbyist forum might require. Namely:

1. Templates. The ability to include the content of another dynamic post into any of your posts on your entire forum is pretty amazing, when you think about it. The way that Elfmage designed it with NuWiki was that his bbcodes that were introduced with NuWiki could be leveraged throughout the forums, and not just on Wiki pages. This has had the unintended (but awesome) consequences of enabling posters to be really creative about how they reference other posts. If users want to reference documentation that changes over time, using the TEMPLATE bbcode is the perfect way to do that.

2. The other Wiki-related bbcodes, all of which are usable site-wide, are also pretty impressive. For example, using the header bbcodes (e.g., H1, H2, H3) on any of your posts automatically creates a table of contents for your particular post. The DIV bbcodes have enabled users to engage in stylistic artwork on their posts, and for my community - a large-scale role playing and fantasy artwork, music, and writing community - this has really opened up a great deal of how users choose to organize their posts and express themselves creatively.

At a bare minimum, if I were replacing NuWiki, I'd want VaultWiki to have these two features. I have confirmed with pegasus previously that the template bbcode only functions on the Wiki-type forums, so that is something that I've had to keep in mind. I'd be curious to know whether there are any site-wide bbcodes that can be used that would bridge wiki functionality to non-wiki forums and posts.

3. Finally, performance. This is the big one, and why I posted this thread to begin with. I tried VaultWiki last year, and, honestly, it was a mess. There were numerous malformed or inefficient queries that brought my site (which was running on a dual-server dedicated configuration with around 200 concurrent users online at the time) screaming to a halt. The NuWiki-to-VaultWiki import script caused a catastrophic corruption of my database and led to, in effect, an unrecoverable database which we had to roll-back to the backup we had created prior to installing VaultWiki. Many of the problems we faced stemmed from some elementary coding oversights (for example, issuing TABLE-wide queries, rather than propagating these queries out by steps or sub-queries).

Pegasus, to his credit, was very attentive and responsive. We were reporting the errors (many of them typos on the import script, or eventualities he had not considered or tested), and he would respond within a few minutes with the correction. I am very appreciative he was there to guide us through using his product, but after all was said and done, there were too many errors that caused us not to be able to use the script. This endured for several hours and after all was said and done, we decided to push back away from this project and deemed it "not ready". That was a year ago.

My biggest concern, aside from some of the missing board-wide features and functionality, has been with the performance of VaultWiki, and while we had a very bad first impression of the import script and how VaultWiki itself handled and rendered Wiki content on even a moderately-busy site, I would be willing to consider VaultWiki again if we could get more detailed information on the performance of VaultWiki, its features and functionality in-depth, and, ideally, some testimonials or examples of how VaultWiki performs on Big Boards or Enterprise sites. I know that a year ago VaultWiki was not ready for prime-time or release, but it is my hope that perhaps it is ready for larger sites and commercial distribution today.

[pegasus]

Hi, I'll try to address as many of the points that Kaelon brought up as I can.

Templates

Templates were instituted on wiki-forum-only basis in VaultWiki because of design flaws we noted with the way NuWiki had handled templates. Of these, the most important were: not being able to cache posts with templates, and calling the BB-Code parser recursively. In long threads with multiple templates, we believed this could make threads very slow.

We were hesitant to make any changes to the way templates worked until a major version change. Since we've reached one now, and many big boards have managed to run NuWiki with these features now for years without problems, we'll try to put this functionality back in the next beta.

BB-Codes

Because we wanted VaultWiki to have a minimal effect on the performance of normal threads, very few BB-Codes were available outside of the wiki. Currently, H=2-7, HR, TABLE, and IMAGE and wiki links work outside the wiki. We are considering adding support for more (TEMPLATE, FOOTNOTE, REFLIST, TOC) starting with the next beta.

The TABLE BB-Code is in a different format than NuWiki's. We were hesitant to add DIV and SPAN codes early on for copyright reasons. Both would have been easy for someone to release as a free modification anyway...

Performance

Performance was one of the reasons we moved away from NuWiki in the first place. However, as you pointed out, there were quite a number of performance issues with VaultWiki early on as well. Since your incident in particular, I have been very vocal and mindful in development with items that could have show-stopper effects on performance.

One of the most obvious changes is something you mentioned, propagating data changes in steps. After your incident, the import scripts were rewritten to do this. Last January we completely rewrote the install/upgrade system to do this. Data management involving multiple tables has been designed to do this whenever possible.

In terms of MySQL optimization, I am always looking for ways to improve queries, whether it be by JOINs or WHERE clauses, or if it means adding a second query to reduce the load and allow using a faster index. Because we were designed for vBulletin 3.7 and also support 3.6, we try to avoid using sub-queries completely to keep the same requirements. Most recently for 3.0.0 Beta 1, we have removed COUNT queries completely that were only used to determine the total number of pages during upgrades and imports.

[InformationNirvana]

The ability to include the content of another dynamic post into any of your posts on your entire forum is pretty amazing, when you think about it.

Is this feature the same feature I had discussed here ?

[pegasus]

I believe it is. Since Kaelon's been running a big board fine with this feature for years, I think it's proved itself to not be uber-damaging to thread performance. I guess it's about time we put this in. I'll update the new-feature queue with this item and see what happens.

[Alfa1]

I'm in the same boat as Kaelon. Performance is the main issue. I would need to see it working on a high traffic board, before I can consider Vaultwiki.
With 5-10k concurrent users and millions of page views, I need a replacement for Nuwiki that performs just as well. Previously my impression was that Vaultwiki was not up to the task, but meanwhile there has been a lot of development and time passed, so I am open to find out.

I would appreciate it if people that are running busy sites with Vaultwiki would contact me.

[InformationNirvana]

With 5-10k concurrent users !!
O M G.

[Alfa1]

Concurrent meaning online users within the last 5 minutes. i.e. showing up in the users online box. Not concurrent as requesting data at the same second.
Still these numbers need bulletproof software. NuWiki is able to cope with it. If Vaultwiki is as well, then that would be great. Any example of large or high traffic sites would be welcome.

[InformationNirvana]

I think most software would fail at those thresholds. I bet you'd need one helluva good server to run Mediawiki for that number of users.
It would be interesting to see if vaultwiki would work on sites that big but not sure that a small product like vW should target such a small market (super high volume sites).

[Alfa1]

On top of vbulletin, I run 65 vbulletin add-ons, including large add-ons, NuWiki and my site is running fast on 1 high end server. Most software works well at those thresholds, but if software is buggy, then it will normally become apparent quickly, as my 677.000 monthly readers will quickly try out all available features.

However, trying out Vaultwiki without any indication that it works on large sites doesn't seem an option, as it means converting NuWiki to Vaultwiki. Going back to a backup means loosing the content added after the backup date. The high number of users also means many posts per day.

[kmike]

So what's been the word on the big/active boards using VaultWiki? Alfa1, have you purchased it?

Another question - those who are willing to migrate from NuWiki, why do you want to do it? Just the lack of support? What are the features of VaultWiki that make the migration attractive?

[pegasus]

I don't think any Big-Board admins are perusing threads like these. Most users here just log in when they need to download a new release. We will be sending out our first community newsletter this month, so this topic will be something I'll be sure to mention - maybe it will get a little more attention that way.

There are plenty of reasons to migrate from NuWiki. To start, the lack of support is a big reason, but also that no development is being done on the project, so it will NOT be fixed. NuWiki did not work properly with vBulletin 3.8, but not at all with 4.0. I reported early in 2007 that NuWiki had problems with inline moderation in 3.7, and was not extensible.

Features VaultWiki has that NuWiki did not:

• Special pages - statistic reports for the wiki
• Namespace support - NuWiki was limited in that you could only have 1 linkable page with a given title
• Its own URLs - NuWiki used showthread URLs
• AJAX section editing - rather than editing the whole page, you can edit just a paragraph via AJAX
• Multiple levels of page protection - NuWiki had an almost redundant function to open/close thread
• Book pages and nested book support - NuWiki had books but they weren't pages and couldn't be nested
• A style that doesn't look like forum posts
• Categories - the ability to organize pages similar to a tagging system
• Reusable wiki-image assets
• Social group pages
• Ability to translate pages in multiple languages
• Hooks so coders can write mods
• and many more I'm forgetting

There are some things VaultWiki does not have yet that were in NuWiki, and some users have these among their reasons for not migrating... yet:

• Ability to use Templates in ANY forum post - early on we deemed this bad for performance if done correctly (NuWiki's system was better on performance but had some order-of-operations bugs)
• Ability to create styled divs and spans - but this could easily be implemented as a standalone modification
• A "traditional" TABLE bb-code (also had TR, TD, etc - VaultWiki uses the piped format)
• A BB-Code that returned a list of all available pages - we deemed this bad for performance also, but could easily be implemented as a standalone modification

[kmike]

Well, how to get feedback from the big boards then, if you are reluctant to disclose the "big" customers using VaultWiki?
Let's try it from another angle: is VaultWiki installed on any board with active users in peak time (as measured by vBulletin) over 2000? Or at least over 1000? Knowing this will be a big testimonial to your performance improvements claims. Because, speaking frankly, the initial posts in this thread are very discouraging - poor handling of even 200 users online which is nowhere near the level of activity of the big boards.

Another question - I got the impression that vB4 version of VaultWiki will have some features not available in vB3 version. Is that right? What is the difference?