Upgrade wizard dies silently in 6.2.1

[mgaudette]

Hi,

I'm trying to update to 6.3.1, but I can't even get to the upload step. The Upgrade Wizard dies after or during the system check, with a blank screen. The top menu is still there (and working) but the bottom part is all blank.

I do get the "system check in progress" widget, but it dissapears and then noithing.

My log files have nothing in them except for upgradeWizard.log which has:

Mon, 30 Jan 2012 13:44:40 -0500 [UpgradeWizard] - -----------------------------------------------------------------------------
Mon, 30 Jan 2012 13:44:40 -0500 [UpgradeWizard] - Upgrade started. At start.php
Mon, 30 Jan 2012 13:44:40 -0500 [UpgradeWizard] - at unlinkTempFiles()
Mon, 30 Jan 2012 13:44:40 -0500 [UpgradeWizard] - finished!
Mon, 30 Jan 2012 13:44:40 -0500 [UpgradeWizard] - resetting $_SESSION
Mon, 30 Jan 2012 13:44:42 -0500 [UpgradeWizard] - setting session variables...
Mon, 30 Jan 2012 13:44:42 -0500 [UpgradeWizard] - [At systemCheck.php]
Mon, 30 Jan 2012 13:44:42 -0500 [UpgradeWizard] - Starting file permission check...
Mon, 30 Jan 2012 13:45:38 -0500 [UpgradeWizard] - File [/var/www/html/whatever/sugarcrm9.log] not writable - saving for display

Where should I look next? I think this is my first upgrade since migrating the CRM to a different machine, but everything else works well....

[Chris_C]

Hi,

I'm trying to update to 6.3.1, but I can't even get to the upload step. The Upgrade Wizard dies after or during the system check, with a blank screen. The top menu is still there (and working) but the bottom part is all blank.

I do get the "system check in progress" widget, but it dissapears and then noithing.

My log files have nothing in them except for upgradeWizard.log which has:




Where should I look next? I think this is my first upgrade since migrating the CRM to a different machine, but everything else works well....

I've put together a PHP script that just might fix this permissions problem.

It's attached.

Put it in the root of your sugar instance, and go to the URL in your browser.

Post back with your results!

[mgaudette]

Thanks, in the end I used the silent upgrader and it works flawlessly.

I've been using SugarCRM since v2 or 3 I think, and I can`t say the upgrade process, in terms of reliability, is very impressive. There definitely needs to be more checking of files, permissions or prerequisites (and clear information given when a failure occurs).

As it stands, I have no idea what was wrong, and no idea why it worked with the silent upgrader.

But it`s fixed!

[Chris_C]

Thanks, in the end I used the silent upgrader and it works flawlessly.

I've been using SugarCRM since v2 or 3 I think, and I can`t say the upgrade process, in terms of reliability, is very impressive. There definitely needs to be more checking of files, permissions or prerequisites (and clear information given when a failure occurs).

As it stands, I have no idea what was wrong, and no idea why it worked with the silent upgrader.

But it`s fixed!

I agree - the upgrade process needs to be bulletproof.

Same thing with module loader - bulletproof as related to module upgrades.

There should be a Sugar Update. Sugar should automatically go online and check for updates every 24 hours - for Sugar itself, and for all loaded modules. Then it should notify the Admin, and give the option of what to update. With a flawless "undo" if it doesnt work for any reason.

[johnwbyrd]

I agree - the upgrade process needs to be bulletproof.

Same thing with module loader - bulletproof as related to module upgrades.

There should be a Sugar Update. Sugar should automatically go online and check for updates every 24 hours - for Sugar itself, and for all loaded modules. Then it should notify the Admin, and give the option of what to update. With a flawless "undo" if it doesnt work for any reason.

The Upgrade Wizard will fail silently when uploading patches unless you fix your php.ini files as described here:

SugarCRM Developer Blog » Blog Archive » HOWTO: Get around file upload size limit issues

Fantastically annoying, yes, and SugarCRM should check this before even permitting the Upgrade Wizard to run.

Don't expect this bug to be fixed or acknowledged.

[jmertic]

Actually, if you want to file a bug on it at SugarCRM Bug Tracker | Open Source Business & Social CRM - SugarCRM or provide a code fix at https://github.com/sugarcrm/sugarcrm_dev/, I will personally make sure it gets addressed as soon as possible.

Thanks!

[Chris_C]

Actually, if you want to file a bug on it at SugarCRM Bug Tracker | Open Source Business & Social CRM - SugarCRM or provide a code fix at https://github.com/sugarcrm/sugarcrm_dev/, I will personally make sure it gets addressed as soon as possible.

Thanks!

I"ll post the solutions here for all to see and comment on, then submit the bug to the Sugar Bug Tracker for the Sugar engineers to add to Sugar CE.

First, the Joomla open source PHP application framework community has already solved this issue (upload and post size limits) years ago.

HEre are the solutions Joomla devs came up with and/or borrowed from other open source PHP application frameworks:

1) Joomla comes out of the box with a Flash uploader option - and a simple radio button in the admin interface to enable it.

Typically this Flash uploader gets used in the Joomla media manager for uploading large (greater than 2MB) pics, videos and audio files.

You can also set Joomla to use the flash uploader for uploading the ZIP packages for manually installing and upgrading Joomla extensions (called modules in Sugar).

The Flash uploader totally sidesteps Apache's upload_max_file_size and post_max_size (2M default) because Flash doesnt use ONE big POST (upload) mechanism of the http (web) server. It chunks it down into several small POSTs - maybe 64 KB at a time. Briiliant.

The major drawback is: iDevices do not support Flash, so this Flash uploader fails to run on the iPad and iPhone - so any file uploads greater than those 2 limits will fail. This really is awful because you lose system administration abilities while "mobile"/away from your desktop (ie, while on the iDevices).

For this reason - I recommend that Sugar NOT use the Flash uploader.

2) Likewise - I recommend that Sugar NOT use the Java uploader - it's not compatible with iphone/ipad - and the number of users of those devices is too huge to not support.

3) Uber-uploader (runs in perl on the server) , and.or Plupload (javascript + dynamic detection of front end flash or html5) is what I recommend Sugar use for large file upload (Upgrade Wizard, Import Wizard, Module Loader). Uber-upload runs in Perl, on the server. And all Linux servers have Perl support (must be added to windows servers, but since Perl is used by nearly all of the popular CGI scripts, Perl often had already been installed on the windows server, by the time you install Sugar ).

These two, uber-uploader and plupload, are widely used by other open source projects for solving exactly this problem of HTTP POST timeouts interrupting large file uploads. Also solves the problem of reaching the 2M default post_max_size limit.

Same as the Flash and Java uploaders, uber-upload/Plupload allow you the user/admin to upload HUGE files (it easily lets you upload a gigabyte file size) in small chunks of around 64KB.

No need for Sugar's installer app code, nor the Sugar system admin, to ever change the Apache parameter max_upload_file_size, nor the post_max_size, as long as Sugar would use uber-upload or Plupload for all places where large files are uploaded: Upgrade Wizard, Import Wizard, and Module Loader.

Plupload and uber-upload both just work. One or both should be added to Sugar as the default methods for uploading files in Upgrade Wizard, Import Wizard, and Module Loader.

[Chris_C]

Actually, if you want to file a bug on it at SugarCRM Bug Tracker | Open Source Business & Social CRM - SugarCRM or provide a code fix at https://github.com/sugarcrm/sugarcrm_dev/, I will personally make sure it gets addressed as soon as possible.

Thanks!

John,
HEre is the bug 51845 I just filed :
SugarCRM Bug Tracker | Open Source Business & Social CRM - SugarCRM
Could you have a look and make sure it gets addressed ASAP, please.

Does the Sugar dev team realize that, because they didnt engineer into Sugar a large file uploader component, Sugar has been crashing for hundreds and thousands of users since about 2006. Evidence all over the forum, in numerous reports from plenty of users ripping their own hair out....

[jmertic]

Thanks Chris, I am following up internally on this now.

Would having better error messaging be a reasonable stop-gap measure until we can figure out a better way to implement this?

[Chris_C]

Thanks Chris, I am following up internally on this now.

Would having better error messaging be a reasonable stop-gap measure until we can figure out a better way to implement this?

John,
MAYBE. Better error messaging MIGHT be a reasonable stop-gap measure. The requirement is, the error messaging must take place up-front, BEFORE Sugar gets itself into an unrecoverable state.

More specifically, to satisfy this requirement, the code would need to do this:
1) query the web server, in a web-server-independent way (remember - Sugar supports no just Apache web server - Sugar also runs on Microsoft IIS web server, as well... and possibly new-generation web servers such as nginx) and dynamically detect that the web server is running with lower than recommended settings for max_upload_file_size and post_max_size (or their equivalents on IIS and nginx).

THe reason these 2 settings must be detected dynamically is : they come from 2 different possible locations.
On a dedicated server: from php.ini OR .htaccess
On a shared server : from .htaccess only, generally speaking the user will be forbiden from modifying php.ini

2) when too low a file size is detected for either of these 2 file-upload maximum size settings, don't even present the user with a "file upload" behavior, the ole "click Browse and select the file to upload". Instead, show the user a link to specific instructions of which 2 lines to add, to either the php.ini or .htaccess file. And the 2 lines are different depending on whether they are to go into .htaccess or php.ini:

if you modify php.ini, it requires the following syntax:

upload_max_filesize = 100M
post_max_size = 100M

if you modify .htaccess, it requires the following syntax:

php_value upload_max_filesize 100M
php_value post_max_size 100M



Side note - I find Sugar 6.4 and below makes far too many assumptions of the runtime environment being correctly configured, so that when it is fatally misconfigured, Sugar fails so terribly that it's a catastrophe.

Sugar needs to raise its quality and exceed common industry standards for reporting/catching/automatically recovering from serious system errors that would otherwise put Sugar into a user-non-recoverable state - I call it that, because a normal user (sales/marketing/business person) does not have the skill set to recover from that kind of an error state.

THink of what the typical user is faced with: they go to load a new module they just bought and are excited to try, or they're in a rush to import data with the Sugar Import Wizard, or anxious to upgrade Sugar to the latest security patch with the Upgrade Wizard.

Any of these three will FAIL horribly because of obscure upload file size limits not being set correctly in a config file.

All this because Sugar is using 1995 technology to upload the file - HTTP POST.

Only a trained Sugar system admin could manually recover from that type of error state, since they would have to use the operating system command line, and possibly a database admin tool such as phpMyAdmin. They have to find out which files to delete, so that Sugar can get "unstuck" from the step where the upload failed, and go back to the beginning of the wizard and try again.

The only real , effective , suitable solution to this huge problem, which has cost users thousands of hours of frustration, and possibly even lost customers to Sugar , and probably also caused Sugar users to suffer losses, is to replace all HTTP POST file uploads with a modern (2003 and newer) small-chunk type of file upload technology. Any of these 3 are fantastic, reliable, and open-source licensed:

1) Plupload,
2) uber_uploader,
3) jQuery File Upload.