Issue: Error after changing thumbnail sizes

Issue Tools
- View Changes

October 19, 2023 2:20 AM

mazzly

Junior Member

Error after changing thumbnail sizes

Code:

ErrorException: Job vw\vw\Handler\Job\Defer: [E_WARNING] Undefined variable $thumblist

Code:

#0 src/addons/vw/vw/_core/model/cron/deferred/settings/thumbnails/vw.php(81): XF::handlePhpError(2, '[E_WARNING] Und...', '/home/nginx/dom...', 81)
#1 src/addons/vw/vw/_core/model/cron/deferred/settings/vw.php(45): vw_Cron_Deferred_Settings_Thumbnails_Model->rebuild(Array, 600)
#2 src/addons/vw/vw/_core/model/cron/deferred/vw.php(66): vw_Cron_Deferred_Settings_Model->run(Array)
#3 src/addons/vw/vw/Handler/Job/Defer.php(41): vw_Cron_Deferred_Model->exec()
#4 src/XF/Job/Manager.php(260): vw\vw\Handler\Job\Defer->run(8)
#5 src/XF/Job/Manager.php(202): XF\Job\Manager->runJobInternal(Array, 8)
#6 src/XF/Job/Manager.php(86): XF\Job\Manager->runJobEntry(Array, 8)
#7 job.php(43): XF\Job\Manager->runQueue(false, 8)
#8 {main}

Code:

array(4) {
  ["url"] => string(8) "/job.php"
  ["referrer"] => string(76) "https://www.....com/admin.php?options/groups/vw_server_settings/"
  ["_GET"] => array(0) {
  }
  ["_POST"] => array(0) {
  }
}

Also, now our wiki seems to be disabled due to a deferred task, is there some way to see the status of this deferred task? Or e.g. php cmd.php command to run these a bit faster on the server CLI?

Issue Details

Issue Number 6394

Issue Type Bug

Project VaultWiki 4.x Series

Category Attachments

Status Fixed

Priority 4 - Warnings / Script Errors

Affected Version 4.1.7

Fixed Version 4.1.8

Milestone (none)

Software DependencyXenForo 2.x

License TypePaid

Users able to reproduce bug 0

Users unable to reproduce bug 0

Attachments 0

Assigned Users (none)

Tags (none)

October 19, 2023 2:21 AM

mazzly

Junior Member

And now the Wiki settings shows:

The wiki currently appears as disabled because a deferred task has failed. Please check your Server Error Logs to determine the source of the problem. If you cannot find an issue in the logs, retry the failed task.

Retrying the failed task doesn't seem to help...

How can I get it to not be disabled now that it has gone into this "broken" state?

Reply
October 19, 2023 9:28 AM
pegasus

VaultWiki Team
This occurs when rebuilding thumbnails after changing the UHD-related settings, while the base thumbnail size was not changed. It occurs when the rebuild is actually complete, or when there are no thumbnails that need to be rebuilt.
In src/addons/vw/vw/_core/model/cron/deferred/settings/thumbnails/vw.php, find:

Code:

while ($thumb = vw_DB::get()->func()->fetch_array($thumbs))

BEFORE it, add:

Code:

$thumblist = array();

Since your rebuild is probably complete, run the following MySQL query:

Code:

DELETE FROM vw_defer WHERE defer = 'settings' AND hold = 1

Then visit AdminCP > Setup > Options > VaultWiki: Site Config again, and the wiki should reactivate itself.

The error first appeared in 4.1.0 Alpha 1, when this task was introduced.
Reply
October 19, 2023 10:45 AM

mazzly

Junior Member

Okay, that seems to have worked (although the SQL command didn't work) but doing "retry" from the options made it go away.

Is there some way to trigger the rebuilding of the thumbnails? It seems that at least one image hasn't gotten the new size generated, so the ?x=newvalue loads the full image, while the ?x=oldvalue still renders old size correctly..

Reply
October 19, 2023 10:51 AM

pegasus

VaultWiki Team

The deferred task doesn't actually create any thumbnails. It clears the list of existing thumbnails so that the next time a thumbnail is requested, it has to be regenerated.

You can force this process manually by deleting everything in the table vw_attachmentthumb, but you may have some stale file data left over that would be difficult to clean. The only safe way I see is to change the value of the thumbnail size setting, which individually cycles through all the entries and cleans the files out.

For the images that don't have new thumbnails, is the full image actually smaller than the new maximum thumbnail size? We don't generate images larger than the full image.

Reply
October 20, 2023 2:11 AM

mazzly

Junior Member

You can force this process manually by deleting everything in the table vw_attachmentthumb, but you may have some stale file data left over that would be difficult to clean. The only safe way I see is to change the value of the thumbnail size setting, which individually cycles through all the entries and cleans the files out.

Okay so I guess I'll have to change it to some similar value then to get rebuild...

For the images that don't have new thumbnails, is the full image actually smaller than the new maximum thumbnail size? We don't generate images larger than the full image.

The full image I found is larger that what the thumbnail was..

-------

Another thing I noticed is that the thumbnail rebuild seems to have cleared the "views" column data to be 0, it would probably be nice if a thumbnail rebuild didn't remove that data (since it is basically the same thumbnail, just different resolution)

Reply
October 20, 2023 2:23 AM

mazzly

Junior Member

Okay, after changing the value a couple of times I can see the xf_vw_attachmentthumb table finally being "flushed"..

The image in question still doesn't render correctly (it is full size)

And now when checking another page the thumbnail is rendered "broken", i.e. 95% of the image is black..

Reply
October 20, 2023 8:14 PM

pegasus

VaultWiki Team

For the image in question that is showing the full image instead of a thumbnail, the math says that VaultWiki thinks you need at least 78,532,480 bytes of memory usage allowed in order to successfully create a thumbnail based on that image (most likely it's more depending on actual memory usage on your forum). If there is not enough memory available, VaultWiki will remember that result and only reply with the full image.

So let's first check whether this happened to any images. See if the table vw_attachmentfail has any entries.
If it does, let's see whether PHP's memory_limit is high enough. Also, if your PHP installation blocks access to ini_get or otherwise won't allow reading of the memory limit due to other security measures you may have, VaultWiki will assume your memory_limit is extremely low.

If you increase your memory_limit, you can get VaultWiki to reattempt by clearing out the vw_attachmentfail table. This should be safe as it's just a logging table.

As for the thumbnail that came out black, I would think this is an issue with the underlying image library that PHP uses, that may have crashed on that image. But it is possible that VaultWiki corrupted the output image when it copied the image metadata to the thumbnail, if it didn't handle particular metadata properly. I will have to run some local tests on the image in question.

I think in the future we need to add a moderator option to image-pages and image-history to clean the existing thumbnails for an image, for example if a thumbnail comes out black, to try again.

Reply
October 21, 2023 4:08 AM

mazzly

Junior Member

So let's first check whether this happened to any images. See if the table vw_attachmentfail has any entries.
If it does, let's see whether PHP's memory_limit is high enough. Also, if your PHP installation blocks access to ini_get or otherwise won't allow reading of the memory limit due to other security measures you may have, VaultWiki will assume your memory_limit is extremely low.

xf_vw_attachmentfail has 111 rows, but not any is with attachfileid matching the image in question...

PHP memory_limit is 1280M and should be enough what I can tell, but we could always increase it if you think more would be needed?

I tested printing ini_get('memory_limit') with one of my own add-ons and it worked, so nothing should be blocking that one..

As for the thumbnail that came out black, I would think this is an issue with the underlying image library that PHP uses, that may have crashed on that image. But it is possible that VaultWiki corrupted the output image when it copied the image metadata to the thumbnail, if it didn't handle particular metadata properly. I will have to run some local tests on the image in question.

I noticed that our forum was set to use "PHP built-in GD image library" instead of "Imagemagick PECL extension" which should be better from what I've gathered, so it might've been related to that? :thinking:

Reply
October 21, 2023 4:33 AM

mazzly

Junior Member

Is there some way I can trigger the regeneration of that single file in db?

I tried removing the row from xf_vw_attachmentthumb where attachmentid matched the broken one, but maybe those are different ids?

Reply
October 22, 2023 12:33 PM
pegasus

VaultWiki Team
I do not recommend having your memory limit that high. Unless you have a particular need, 256M should be high enough for most purposes and should balance the need for sub-4K image processing with reducing the severity of slow loris attacks. I have responded to your other report about $resized errors, which I believe may also cover the black images.

What I am still wondering about though is the vw_attachmentfail entries. Since you have enough memory, I wonder if you are getting errors somewhere else. In src/addons/vw/vw/_core/controller/asset/vw.php, find:

Code:

$errors = $thumbdm->get_errors();

AFTER it, add:

Code:

if ($errors) XF::logException(new Exception(implode('... ', $errors)));

Then clear out your vw_attachmentfail table and wait for the log entries in XenForo's server error log.
Reply

October 22, 2023 2:48 PM

mazzly

Junior Member

It seems I had to revert the changes from https://www.vaultwiki.org/issues/6395/ to get any errors to trigger..

Code:

Exception: Failed to generate a thumbnail. src/addons/vw/vw/_core/controller/asset/vw.php:823
Generated by: mazzly Oct 22, 2023 at 9:45 PM

Code:

#0 src/addons/vw/vw/_core/controller/asset/vw.php(130): vw_Asset_Controller->create_thumbnail(Array)
#1 src/addons/vw/vw/Pub/Controller/Wiki/Asset.php(43): vw_Asset_Controller->resolve()
#2 src/addons/vw/vw/Mvc/Dispatcher.php(30): vw\vw\Pub\Controller\Wiki\Asset->_actionIndex(Object(XF\Mvc\ParameterBag))
#3 src/addons/vw/vw/Pub/Controller/Wiki/AbstractController.php(26): vw\vw\Mvc\Dispatcher::dispatch(Object(vw\vw\Pub\Controller\Wiki\Asset), 'index', Object(XF\Mvc\ParameterBag))
#4 src/XF/Mvc/Dispatcher.php(352): vw\vw\Pub\Controller\Wiki\AbstractController->actionIndex(Object(XF\Mvc\ParameterBag))
#5 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('vw\\vw:Wiki\\Asse...', 'Index', Object(XF\Mvc\RouteMatch), Object(vw\vw\Pub\Controller\Wiki\Asset), NULL)
#6 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(vw\vw\Pub\Controller\Wiki\Asset), NULL)
#7 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#8 src/XF/App.php(2487): XF\Mvc\Dispatcher->run()
#9 src/XF.php(524): XF\App->run()
#10 index.php(20): XF::runApp('XF\\Pub\\App')
#11 {main}

Code:

array(4) {
  ["url"] => string(26) "/wiki-asset/?pid=525&x=450"
  ["referrer"] => string(42) "https://www.....com/wiki/Ainur"
  ["_GET"] => array(3) {
    ["/wiki-asset/"] => string(0) ""
    ["pid"] => string(3) "525"
    ["x"] => string(3) "450"
  }
  ["_POST"] => array(0) {
  }
}

October 23, 2023 9:01 AM
pegasus

VaultWiki Team
In src/addons/vw/vw/config.php, add the following:

Code:

define('VW_DEBUG_ATTACH', 1);

This should give a more verbose reason for that particular error.

Also, in your php.ini, make sure that opcache.fast_shutdown is NOT turned on. That can cause weird problems like ending the PHP script and running shutdown functions before work is actually done.
Reply

October 23, 2023 12:55 PM

mazzly

Junior Member

Okay now we're getting somewhere

Code:

ImagickException: time limit exceeded `/home/nginx/domains/.../public/internal_data/vw_attachments/7/f/271/full.vw' @ fatal/cache.c/GetImagePixelCache/1867 src/addons/vw/vw/_core/model/image/format/vw.php:1029
Generated by: mazzly Oct 23, 2023 at 7:53 PM

Code:

#0 src/addons/vw/vw/_core/model/image/format/vw.php(1029): Imagick->resizeImage(450, 689, 11, 1, true)
#1 src/addons/vw/vw/_core/controller/dm/attach/thumb/vw.php(307): vw_Image_Format_Model->resize(450, 689, 'jpg')
#2 src/addons/vw/vw/_core/controller/dm/attach/thumb/vw.php(131): vw_DM_Attach_Thumb_Controller->create_thumbnail('jpg', Object(vw_Image_Format_JPG_Model), 450, 850)
#3 src/addons/vw/vw/XF/Entity/EntityBase.php(513): vw_DM_Attach_Thumb_Controller->pre_save(true)
#4 src/XF/Mvc/Entity/Entity.php(1366): vw\vw\XF\Entity\EntityBase->_preSave()
#5 src/addons/vw/vw/XF/Entity/EntityBase.php(504): XF\Mvc\Entity\Entity->preSave()
#6 src/addons/vw/vw/_core/controller/dm/xf2.php(387): vw\vw\XF\Entity\EntityBase->vwPreSave()
#7 src/addons/vw/vw/_core/controller/asset/vw.php(813): vw_DM_Controller_XF2->pre_save()
#8 src/addons/vw/vw/_core/controller/asset/vw.php(130): vw_Asset_Controller->create_thumbnail(Array)
#9 src/addons/vw/vw/Pub/Controller/Wiki/Asset.php(43): vw_Asset_Controller->resolve()
#10 src/addons/vw/vw/Mvc/Dispatcher.php(30): vw\vw\Pub\Controller\Wiki\Asset->_actionIndex(Object(XF\Mvc\ParameterBag))
#11 src/addons/vw/vw/Pub/Controller/Wiki/AbstractController.php(26): vw\vw\Mvc\Dispatcher::dispatch(Object(vw\vw\Pub\Controller\Wiki\Asset), 'index', Object(XF\Mvc\ParameterBag))
#12 src/XF/Mvc/Dispatcher.php(352): vw\vw\Pub\Controller\Wiki\AbstractController->actionIndex(Object(XF\Mvc\ParameterBag))
#13 src/XF/Mvc/Dispatcher.php(259): XF\Mvc\Dispatcher->dispatchClass('vw\\vw:Wiki\\Asse...', 'Index', Object(XF\Mvc\RouteMatch), Object(vw\vw\Pub\Controller\Wiki\Asset), NULL)
#14 src/XF/Mvc/Dispatcher.php(115): XF\Mvc\Dispatcher->dispatchFromMatch(Object(XF\Mvc\RouteMatch), Object(vw\vw\Pub\Controller\Wiki\Asset), NULL)
#15 src/XF/Mvc/Dispatcher.php(57): XF\Mvc\Dispatcher->dispatchLoop(Object(XF\Mvc\RouteMatch))
#16 src/XF/App.php(2487): XF\Mvc\Dispatcher->run()
#17 src/XF.php(524): XF\App->run()
#18 index.php(20): XF::runApp('XF\\Pub\\App')
#19 {main}

Code:

array(4) {
  ["url"] => string(26) "/wiki-asset/?pid=525&x=450"
  ["referrer"] => string(42) "..."
  ["_GET"] => array(3) {
    ["/wiki-asset/"] => string(0) ""
    ["pid"] => string(3) "525"
    ["x"] => string(3) "450"
  }
  ["_POST"] => array(0) {
  }
}

Also, interestingly, this doesn't show up in xf_vw_attachmentfail...

October 23, 2023 1:11 PM

mazzly

Junior Member

Also, during above error nothing was logged in PHP slowlog, which log anything over 1s

Reply
October 23, 2023 2:35 PM

pegasus

VaultWiki Team

You won't get a vw_attachmentfail now because we threw an exception instead.

Based on the error message, your server was unable to generate a thumbnail for this within 5 seconds. I don't think we should increase the time limit, or the D/DoS vector gets wider. This is probably due to trying to generate too many thumbnails at once (maybe loading a page that needs a lot of thumbnails). In the future we can track whether vw_attachmentfail was due to time limit, and possibly retry those images again during a lower traffic period. Also we can add a sort of queue, where we only let a certain number of thumbnails generate at once in order to avoid them causing each other to time out (probably what is happening). If there's no room in the queue, we can display the full version and not issue vw_attachmentfail until it has a chance to go through queue.

The only thing would be figuring out what an appropriate queue size would be. It's probably something like 1G / memory_limit, where 1G could be a config variable.

Reply

+ Reply

All times are GMT -4. The time now is 2:30 PM.

This site uses cookies to help personalize content, to tailor your experience, and to keep you logged in if you register.
By continuing to use this site, you are consenting to our use of cookies.

Learn more… Accept Remind me later

Welcome to VaultWiki.org, home of the wiki add-on for vBulletin and XenForo!

Issue: Error after changing thumbnail sizes

Issue Tools