Slow Access To Module Loader

[jnovak]

I am running SugarCRM 4.5.1a.

Is there a reason it takes a very long time to enter the the Module Loader? I have been upgrading since 3.51. It takes about 5 minutes to enter. When I installed a new copy of SugarCRM 4.5.1 entry was almost immediate.

Any ideas what would slow this process down?

Thanks,
John Novak

[ruchida]

Module Loader inspects all of files under cache/upload/upgrades directory before it starts its task. Your cache directory may have lots of (unnecessary) files that makes Module Loader slow.

Check that directory in your server and delete unnecessary files. You may want to delete at least temp files in upload/upgrades/temp, that will simply improves performance.

[jnovak]

Module Loader inspects all of files under cache/upload/upgrades directory before it starts its task. Your cache directory may have lots of (unnecessary) files that makes Module Loader slow.

Check that directory in your server and delete unnecessary files. You may want to delete at least temp files in upload/upgrades/temp, that will simply improves performance.

How do you determine which files are necessary in cache/upload/upgrades? There are no files in cache/upload/upgrades/temp.

Does this have anything to do with the following error that I get once I get into Module Loader:

Error: PCLZIP_ERR_MISSING_FILE (-4) : Missing archive file 'cache/upload/upgrades/module/tabpanels11b-351a.zip'


Thanks,
John Novak

[ruchida]

Since you're using 4.5.1, SugarCRM is going to connect to SugarExchange when Modules Loader is selected. This happens even when you're going to load a module locally. This is a little bit bad design of SugarCRM.

[jnovak]

OK, but how do you determine which files & folders to delete? Also, what is causing the error I am gitting?

Thanks
John Novak

[majed]

4.5.1 b should include a patch for the issue of the module loader loading slowly once you have loaded several modules. After the points ruchida made we will be doing a full review on the module loader process. I do not believe the auto-connect feature is causing the slow down, but it will be reviewed. Our intention was simply to make it as easy as possible for people to get new modules and upgrades from within the application hence the auto sign-in feature. I will post the code for this fix later today so you guys can give it a whirl and let us know if it fixes the slowdown issue. We use the module loader all the time internally so we know the pains, and thank you for the feedback we always have our eyes peeled to these forums for ways to make the product better.

Also the PCL_ZIP error sounds as though the zip file is missing can you check to make sure cache/upload/upgrades/module/tabpanels11b-351a.zip is there and if so. Pull it down from the server and make sure it is a good zip file. Just unzip it with winzip or unzip and if it unzips then there is another issue, but keep us posted so we can get this resolved.

Expect the code by the end of our workday monday.

Thanks,
Majed

[jnovak]

4.5.1 b should include a patch for the issue of the module loader loading slowly once you have loaded several modules. After the points ruchida made we will be doing a full review on the module loader process. I do not believe the auto-connect feature is causing the slow down, but it will be reviewed. Our intention was simply to make it as easy as possible for people to get new modules and upgrades from within the application hence the auto sign-in feature. I will post the code for this fix later today so you guys can give it a whirl and let us know if it fixes the slowdown issue. We use the module loader all the time internally so we know the pains, and thank you for the feedback we always have our eyes peeled to these forums for ways to make the product better.

Also the PCL_ZIP error sounds as though the zip file is missing can you check to make sure cache/upload/upgrades/module/tabpanels11b-351a.zip is there and if so. Pull it down from the server and make sure it is a good zip file. Just unzip it with winzip or unzip and if it unzips then there is another issue, but keep us posted so we can get this resolved.

Expect the code by the end of our workday monday.

Thanks,
Majed

If this module is not really loaded, is there a way of removing the message? Is there a way to fix the problem so that SugarCRM does not think it is loaded?

Thanks,
John Novak

[majed]

As promised here are the code changes that need to be made


in a file located at include/dir_inc.php

change the function findAllFiles to this

function findAllFiles( $the_dir, $the_array, $include_dirs=false, $ext=''){
    $d = dir($the_dir);
    while( $f = $d->read() ) {
        if( $f == "." || $f == ".." ){
            continue;
        }

        if( is_dir( "$the_dir/$f" ) ) {
			if($include_dirs) {
				$the_array[] = clean_path("$the_dir/$f");
			}
            $the_array = findAllFiles( "$the_dir/$f/", $the_array, $include_dirs , $ext );
        } else {
			if(empty($ext) || strrpos("$f", $ext) == strlen("$f") - strlen($ext)){
				$the_array[] = clean_path("$the_dir/$f");
			}
        }


    }
    rsort($the_array);
    
    return $the_array;
}

This will allow you to filter the files that are being browsed and we are going to use this filter in the file

modules/UpgradeWizard/uw_utils.php. Around line 283 there is a function called getValidPatchName in there change

$upgrade_contents = findAllFiles($base_upgrade_dir, array(), false);

To

$upgrade_contents = findAllFiles($base_upgrade_dir, array(), false, 'zip');

This should boost your performance if not let me know. The patch is checked in, but it was checked in a while ago so i may be missing 1 more change that needs to occur.

As for deleting the zip you should be able to just remove it from the directory. You also will need to delete the manifest.php file.

Thanks,
Majed

[jnovak]

The directory "cache\upload\upgrades\module" did not have anything in it and thats what was giving me the error. When I put the .zip in this directory, the error message disappeared. There is no manifest file in the directory. When I remove the .zip file the error returns. With the .zip file in the directory, the module loader shows the module as being installed. Since this module is not installed, how do I delete the .zip and tell SugarCRM that the module is not loaded?

At one time, many versions ago, I tried to load this module but it is no longer loaded. SugarCRM is still looking for it. Where is the reference that SugarCRM is using such that it thinks it is loaded?

Thanks,
John Novak

[jnovak]

I tried uninstalling the module from the list in module loader and got the following message:


Warning: PackageManager::getPackagesInStaging(cache/upload/upgrades/module/whosonline10c-351-manifest.php) [function.getPackagesInStaging]: failed to open stream: No such file or directory in /var/www/html/sugarcrm/ModuleInstall/PackageManager/PackageManager.php on line 597

Fatal error: PackageManager::getPackagesInStaging() [function.require]: Failed opening required 'cache/upload/upgrades/module/whosonline10c-351-manifest.php' (include_path='.:/usr/share/pear') in /var/www/html/sugarcrm/ModuleInstall/PackageManager/PackageManager.php on line 597

Any idea what this means?

Thanks,
John Novak