Page 1 of 3 123 LastLast
Results 1 to 10 of 23

Thread: Code issues identified in SugarOS 4.5.0h

  1. #1
    glucose is offline Sugar Community Member
    Join Date
    Jan 2007
    Posts
    34

    Default Table Prefixes and Code issues identified in SugarOS 4.5.0h

    I am in the process of modifying SugarOS 4.5.0h to use table prefixes (yes, it is an extremely painful, arduous and tedious process). I'm on day 3 of the process and have put in about 24 hours in so far. It basically amounts to a one-man QA/QC audit of the entire codebase.

    I have identified a number of issues.

    #1) The installer settings confirmation page refuses to acknowledge the user deselecting the automatic updates. I imagine it is related to the exchange of $_REQUEST and $_SESSION variables in the multi-step install process. (I have added a field to the database config for the table prefix to use which is sucessfully saved to the config file and used later in the process) I have not yet taken the time to determine where the problem with the auto-updates setting lies. The corresponding ""send anonymous stats" is not making it into the session variables either from what I can tell, though it is not displayed on the settings confirmation page.

    #2) misspellings (identified thus far):

    Code:
    Found 2 occurrences in 2 files for the word: delted
    
    \data\Link.php (717) 
    	 * many-to-many: delete rows from the link table. related table must have delted and date_modified column.
    
    
    \modules\Users\User.php (117) 
                $q = "SELECT users.id FROM users WHERE users.status = 'Active' AND users.delted = 0 AND users.is_admin = 1";
    
    
    
    
    Found 9 occurrences in 7 files for the word: campaing
    
    \include\language\en_us.lang.php (1255) 
      'campainglog_activity_type_dom' =>
    
    
    \include\language\en_us.lang.php (1268) 
      'campainglog_target_type_dom' =>
    
    
    \modules\CampaignLog\CampaignLog.php (108) 
    	//this function is called statically by the campaing_log subpanel.
    
    
    \modules\CampaignLog\vardefs.php (69) 
    			'options'=>'campainglog_activity_type_dom',
    
    
    \modules\Campaigns\TrackDetailView.php (144) 
    			...$xtpl->assign("MY_CHART", $chart->campaign_response_by_activity_type($app_list_strings['campainglog_activity_type_dom'],$app_list_strings['campainglog_target_type_dom'],$focus->id,$sugar_c...
    
    \modules\Campaigns\TrackDetailView.php (144) 
    			...ign_response_by_activity_type($app_list_strings['campainglog_activity_type_dom'],$app_list_strings['campainglog_target_type_dom'],$focus->id,$sugar_config['tmp_dir'].$cache_file_name,true));
    ...
    
    \modules\CampaignTrackers\CampaignTracker.php (84) 
    	var $relationship_fields = Array('campaing_id'=>'campaign');
    
    
    \modules\Prospects\language\en_us.help.index.html (48) 
    	<li><span class="helpShortcut">Campaigns</span>. Click this option to navigate back to the Campaings Home page from a campaign's detail page.</li>
    
    
    \removeme.php (52) 
    		//	record this activity in the campaing log table..
    Some of these may be intentional, but the query that contains "delted" is definitely in error.

    #3) The performsetup page output is displaying table names from the dictionary array defined in the metadata files as opposed to the actual table names created. (Are these metadata files regenerated at any point after an install? - I get the impression they are... though they seem redundant in that it appears the data they define is already defined in the module class definitions and the vardefs files...)

    I'm not sure about this one now, I just seem to be having a hard time getting the relationships tables built with a table prefix...


    I'm sure I will find more as I go...

    When/If I am successful in getting a copy working properly with table prefixes I will submit it in the event it is desired to incorporate it into future versions. (Gawd I hope the changes in 4.5.1 are not significant, when it is released I intend to back port the changes to my table prefix version of 4.5.0h - I'd hate to have to do that for every new release of SugarOS...)
    Last edited by glucose; 2007-01-25 at 02:46 PM. Reason: updated title to reflect thread content... hmmm though it does nto show in the thread title... something for vBulletin to consider....

  2. #2
    julian's Avatar
    julian is offline Sugar Team Member
    Join Date
    Sep 2004
    Posts
    1,639

    Default Re: Code issues identified in SugarOS 4.5.0h

    I've found that in most cases, $this->table_name is used in queries rather than a hardcoded table-- I'm sure it's hardcoded in a lot of places, but we've began to clean this up.

    If you provided me with a list of places where the table name is hard-coded, and the correct variable to insert in its place, I'd be happy to check in the changes (so no merges are required on your part).

    At that point, prefixes should only be a matter of prepending "x_" to $this->table_name.
    Julian Ostrow
    Systems and Applications Engineer
    SugarCRM Inc.

  3. #3
    glucose is offline Sugar Community Member
    Join Date
    Jan 2007
    Posts
    34

    Default Re: Code issues identified in SugarOS 4.5.0h

    Thanks for the offer, Julian, very kind of you.

    Unfortunately the list of hardcoded table names is huge. I have already made hundreds if not thousands of edits to queries, vardefs, relationship arrays etc. with hardcoded table references and probably have thousands to go. Yes, in some places $this->table_name is used, and it is nice to see, but there are also are a number of cases where code containing queries in classes is called without instantiating the object, requiring a global variable within the function. (e.g. all the build_generic_where_clause functions)

    e.g.: (in Case.php)
    Code:
    	function build_generic_where_clause ($the_query_string) {
    		global $sugar_config;
    		$tbl_prefix = $sugar_config['dbconfig']['db_prefix'];
    	$where_clauses = Array();
    	$the_query_string = PearDatabase::quote(from_html($the_query_string));
    	array_push($where_clauses, $tbl_prefix . "cases.name like '$the_query_string%'");
    	array_push($where_clauses, $tbl_prefix . "accounts.name like '$the_query_string%'");
    
    	if (is_numeric($the_query_string)) array_push($where_clauses, $tbl_prefix . "cases.case_number like '$the_query_string%'");
    
    	$the_where = "";
    
    	foreach($where_clauses as $clause)
    	{
    		if($the_where != "") $the_where .= " or ";
    		$the_where .= $clause;
    	}
    	
    	if($the_where != ""){
    		$the_where = "(".$the_where.")";	
    	}
    	
    	return $the_where;
    	}
    There was also a case of it in Relationship.php that was called during the install without instantiating the object that I had fixed.

    The approach I am taking basically is to put this code into the initialisation function of every class:
    Code:
    		global $sugar_config;
    		$this->table_prefix = $sugar_config['dbconfig']['db_prefix'];
    		$this->table_name = $this->table_prefix . $this->table_name;
    and in more complicated cases (E.g. Case.php):

    Code:
    		global $sugar_config;
    		$this->table_prefix = $sugar_config['dbconfig']['db_prefix'];
    		$this->table_name = $this->table_prefix . $this->table_name;
    		$this->rel_account_table = $this->table_prefix . $this->rel_account_table;
    		$this->rel_contact_table = $this->table_prefix . $this->rel_contact_table;
    		$this->relationship_fields = Array('account_id'=> $this->table_prefix . 'account', 'bug_id' => $this->table_prefix . 'bugs',
    									'task_id'=> $this->table_prefix . 'tasks', 'note_id'=> $this->table_prefix . 'notes',
    									'meeting_id'=> $this->table_prefix . 'meetings', 'call_id'=> $this->table_prefix . 'calls', 'email_id'=> $this->table_prefix . 'emails',									
    									);
    And edit all the hardcoded references in the class to its own table to {$this->table_name} and edit references to other tables in queries in the class by prepending {$this->table_prefix} to them. Many queries using joins require that fields from other tables include the table name, which must also include the prefix... (e.g. modules\Import\UsersLastImport.php has plenty of such queries)


    Another non-table prefix related code problem just found:
    #4) The double quotes in the comments in the string used in the copy of the write_array_to_file function in \modules\InboundEmail\parseEncoding.php (33) need to be escaped like so:
    Code:
        $the_string =   "<?php\n" .
    "\n
    if(empty(\$GLOBALS['sugarEntry'])) die('Not A Valid Entry Point');
    /*********************************************************************************
     * The contents of this file are subject to the SugarCRM Public License Version
     * 1.1.3 (\"License\"); You may not use this file except in compliance with the
     * License. You may obtain a copy of the License at http://www.sugarcrm.com/SPL
     * Software distributed under the License is distributed on an \"AS IS\" basis,
     * WITHOUT WARRANTY OF ANY KIND, either express or implied.  See the License
     * for the specific language governing rights and limitations under the
     * License.
     *
     * All copies of the Covered Code must include on each user interface screen:
     *    (i) the \"Powered by SugarCRM\" logo and
     *    (ii) the SugarCRM copyright notice
     * in the same form as they appear in the distribution.  See full license for
     * requirements.
     *
     * The Original Code is: SugarCRM Open Source
     * The Initial Developer of the Original Code is SugarCRM, Inc.
     * Portions created by SugarCRM are Copyright (C) 2004-2006 SugarCRM, Inc.;
     * All Rights Reserved.
     * Contributor(s): ______________________________________.
     ********************************************************************************/
    /*********************************************************************************
     * Description:
     * Created On: Apr 22, 2006
     * Portions created by SugarCRM are Copyright (C) SugarCRM, Inc. All Rights
     * Reserved. Contributor(s): Chris Nojima
     ********************************************************************************/\n\n"

    The approach you suggest for me to submit these occurrences to you for check-in is just not practical. Because of the way Sugar is written it is a major change to the codebase that affects just about every file.

    Perhaps once complete I could run a diff between stock 4.5.0h files and my table prefix version and send that to you for check in, but I fear that I will be too far behind where 4.5.1 is now...

    EDIT: Basically, I have already done too many undocumented changes... I should have asked for CVS access before I started I guess. I suppose I research that now and see if I can get hooked up with CVS access and at least check out where the code is now and run a few diffs to figure out what I have to make a submission of the edits possible.

    So far I have edited 146 files I only have to go over 2587 more files, assuming there is no database code in the themes (I haven't checked them yet). A list of the files I have edited so far is attached.
    Attached Files Attached Files
    Last edited by glucose; 2007-01-10 at 01:53 AM.

  4. #4
    glucose is offline Sugar Community Member
    Join Date
    Jan 2007
    Posts
    34

    Default Re: Code issues identified in SugarOS 4.5.0h

    Add getUserActions in modules\ACLActions\ACLAction.php to the list of functions called without instantiating the object. (I added a comment to that effect in my edits).

    Whoot! I finally got to the login page running on prefixed tables!!!

    I was stuck for the longest time today because the relationship tables were not being created with the table prefix. I finally clued in that this was because the table dictionary include line in performSetup.php was before the call to handleSugarConfig, Since the metadata files that define the dictionary now require the config to be loaded with the edits that I have made, this was a problem. (I'm a bit baffled why I did not see an undefined index on $sugar_config error in my apache logs indicating this) Anyway, I just moved the include and rel_dictionary definition to after the call to handleSugarConfig et voila!

    Now I need to ensure that the prefixed tables are used throughout the code, find when and how temporary tables are used, ensure they are prefixed as well and then test, test, test... (That will be a bit tricky for me as I have never used SugarCRM before!)

    I'm also considering removing relevant hard coded paths to directories where files are written (i.e. to avoid collisions of multiple instances of Sugar running from one set of files on the server, though this may not be necessary... by now you have probably guessed what my goal is... )

  5. #5
    julian's Avatar
    julian is offline Sugar Team Member
    Join Date
    Sep 2004
    Posts
    1,639

    Default Re: Code issues identified in SugarOS 4.5.0h

    Hello glucose,

    I don't believe we create any temporary tables, although some are created dynamically. When you choose to audit a module, a <module>_audit table is created. When you add custom fields to a module, a <module>_cstm table is created. I don't think there are any other cases of this.
    Julian Ostrow
    Systems and Applications Engineer
    SugarCRM Inc.

  6. #6
    glucose is offline Sugar Community Member
    Join Date
    Jan 2007
    Posts
    34

    Default Re: Code issues identified in SugarOS 4.5.0h

    Thanks for the info julian.

    I have come across the *_cstm custom fields tables in the code and their creation should work with my edits.

    I'm just over halfway through the stock modules at this point (up to Meetings). I hope to finish the modules in the next few days and then move on to the includes. I had already covered all the main modules files but I'm finding plenty of table references in the other module files (often in the ListView, FormBase , Detailview, SearchFields, sometimes even in Menu.php and the dashlets) so I'm going over every single file one by one.

    For the modules that's 1052 files done (not all edited), and 581 files to go. Then the 831 include files and the other non-theme folders. The end is in sight.

    I'm assuming the templates/themes and javascript files have no db tables code in them. (Is this a safe assumption?)

    I see 4.5.1 RC is out, at this point I figure adding the changes in 4.5.1 to my 4.5.0h table prefix version will be minor compared to what I have gone through already in the course of this week.

    The only tricks I haven't quite planned out yet is how to handle people upgrading to a version that handles table prefixes. It will still work fine with no table prefix, I just need to figure out where in the upgrade process to add a null string to the config for the table prefix and perhaps an option somewhere in admin to edit the table prefix should people with to manually change an existing installation to use table prefixes. The other issue is custom module compatibility. I guess for now, older modules will not be compatible with an installation that uses a table prefix, until they are updated to handle it.

    I have not found any other glaring errors in the code along the way so far.

  7. #7
    julian's Avatar
    julian is offline Sugar Team Member
    Join Date
    Sep 2004
    Posts
    1,639

    Default Re: Code issues identified in SugarOS 4.5.0h

    Great, let me know when you have something test-able.
    Julian Ostrow
    Systems and Applications Engineer
    SugarCRM Inc.

  8. #8
    glucose is offline Sugar Community Member
    Join Date
    Jan 2007
    Posts
    34

    Default Re: Code issues identified in SugarOS 4.5.0h

    I did find some temporary tables; in uw_utils.php in the UpgradeWizard module. (I was expecting to perhaps find some fulltext indexed temporary search tables for use with the sql match syntax, but it appears that search strategy is not utilised.)

    I removed the table prefixes I had added to the relationship_fields arrays as quoted above, as these arrays define the field and relationship names as opposed to the field and table names as I had thought. (I have preserved the relationship names as is)

    The UpgradeWizard module is the only one I'm kind of iffy on overall, it needs a closer look. There is also code to run .sql files there which I have not handled, (again, a regex to parse the queries contained therein is a masochistic and unreliable strategy). Not quite sure what to do about that, but if future versions take table prefixes into account then it should not be a concern.

    I skimmed over the includes pretty quick, so its possible I may have missed a few table names there but I think I got them all.

    So here we are one week later and I think I have it finished. ;D

    Something that is bothering me though; I noticed that methods in the UserPreferences class are called with the User object in focus, preventing the use of the $this->table_name syntax in queries in its methods. I half expect this to turn up in other classes where I have edited all the queries to use $this->table_name where possible, so some modules may yet need tweaks to use a {$this->table_prefix}tablename in some places instead, but I have not tested enough yet to determine if this will actually occur.

    I am also a little bothered by the confusion over object vs table names in places (present in the unmodified code). This could lead to additional confusion later, especially with table prefixes implemented.

    I'm going to hold off integrating the changes in 4.5.1 into this table prefix version until 4.5.1 is final. (Unless this looks good enough after testing that you guys want to take care of it and call it 4.5.2)

    I was surprised by the duration and cpu usage of the install when the demo data is included, I can't recall if it was that bad before my modifications, but I seem to recall it was. Installing the demo data may not be possible on some shared hosts due to cpu usage quotas and php execution time limit settings.

    So, without further ado, my 4.5.0.h table prefix version of SugarOS is attached. (LZMA compressed with 7-zip, to save upload time on my dial-up connection, then zipped without compression to allow for upload here under the valid file extensions.)

    http://www.7-zip.org/ (*nix command line version p7zip also available, but judging by some of the code and the line endings in SugarOS I'm guessing most of you are windows users anyway)

    It probably could have been done better, but it seems to work fine here so far. Hopefully I haven't missed much, but I'm sure a few issues will come up and I'd appreciate any help testing it a great deal.

    If you find anything strange, increase the logging level in the log4php.properties and try the action again to capture more info about it. (It would be really nice if the log4php included the file and line number of problem queries, as it is I have to search the entire project for portions to the problem queries to locate them... I could not find an obvious switch for this in log4php, if the feature does not exist perhaps I'll add it at some point)
    Attached Files Attached Files

  9. #9
    glucose is offline Sugar Community Member
    Join Date
    Jan 2007
    Posts
    34

    Default Re: Code issues identified in SugarOS 4.5.0h

    I was getting a "Using $this when not in object context" error in UserPreference.php. Fixed file attached.
    Attached Files Attached Files

  10. #10
    glucose is offline Sugar Community Member
    Join Date
    Jan 2007
    Posts
    34

    Default Re: Table Prefixes and Code issues identified in SugarOS 4.5.0h

    I see that there have been 10 downloads of this table prefixed version (with 5 downloads of the required fixed UserPreference.php, so I guess make that 5 downloads total).

    Does anyone have any issues to report? I have not tested it much yet because I have gotten hung up on the lack of a working products module in SugarOS 4.5.0.h, so progress on my project has been delayed while I try to deal with that, But it does seem to be working fine with the demo data.

    I got osCommerce working with table prefixes as well, but the integration module on SugarForge assumes hardcoded table names even though osCommerce uses constants for table names internally. It also uses a SOAP interface which does not make much sense now that the two can share the same database since they are both using table prefixes with my modifications. (It also requires a working products module for full functionality.) So I'm looking at using Zen-Cart which already uses table prefixes and is much more up to date and active compared to osCommerce. I'm thinking at this point that writing my own Sugar integration module for Zen-Cart would be less work than fixing the osCommerce module to work with table prefixes and a direct database connection instead of the roundabout SOAP interface.

    I'm also curious if the delay of the release of SugarOS 4.5.1 might be related to the inclusion of my table prefix modifications? It would be nice to see 4.5.1 released with table prefix capability, or perhaps a 4.5.1 RC2 for some more testing of it?

    I'm also taking a look at using vtiger as it includes a products module, but it seems to be much less php 5.2.0 / MySQL 5.0.x compatible than SugarOS. (Wow, you have it on the censored words list on these forums!) I do not want to create any bad feelings here. I'm just exploring all my options for the most efficient and logically integrated CRM/Shop setup. I have also taken a look at C3CRM for its products module but it is based on Sugar 4.2.x code and will require much work to be made compatible with SugarOS 4.5.x. I'm kind of wishing I knew all this before I spent a whole week updating SugarOS to use table prefixes, but I think it is still a valuable contribution regardless.

    So, any reports, critiques or suggestions on the table prefix version of SugarOS 4.5.0h?
    Last edited by glucose; 2007-01-25 at 02:48 PM. Reason: spelling

Page 1 of 3 123 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •