Shortlink

Code on GitHub

I have started putting my public code onto GitHub. To get the latest copies of code, please use this link:

https://github.com/opensourceame

I will be documenting the process manager classes soon.

Shortlink

Stash classes online

I’ve made my Stash classes (a lightweight ORM / DBAL of sorts) available for download. The simplest way to explain how they work is through some example code:

 
/*
 * first install the database using the example.sql file that comes with this package
 */
 
define('DSN_STASH_TEST', 'mysql://test:test@localhost/stash_test');
 
require_once 'opensourceame.stash.php';
 
/*
 * This is the singular class for person objects
 */
class person extends stash_bean
{
	protected	$_stash_dsn 		= DSN_STASH_TEST;
	protected	$_stash_table		= 'people';
	protected	$_stash_fields		= array('id', 'first_name', 'last_name');
 
	public function name()
	{
		return $this->last_name.', '.$this->first_name;
	}
}
 
/*
 * This defines the class people as a collection of persons
 */
class people extends stash_collection
{
	protected	$_object_class		= 'person';	
}
 
/*
 * EXAMPLE: fetch an existing person and display the name
 */
 
$person = new person;
$person->retrieve('20fee9bd-5f4d-d34b-5605-4cefaaca32ab');
 
echo $person->name()."\n";
 
/*
 * EXAMPLE: save an archive of the existing record
 * then change the first name from "Will" to "William" and update the record
 */
 
$person->archive();
 
$person->first_name	= 'William';
$person->update();
 
 
/*
 * EXAMPLE: fetching multiple person objects as a collection
 */
$people = new people;
$people->add_condition("and last_name='Smith'");
$people->add_condition("or first_name='Helen'");
$people->retrieve();
 
foreach($people->collection() as $person)
{
	echo $person->name()."\n";
}
Shortlink

Updated logger

I’ve added a zTemplate plugin to my Flexible Logger. This makes logging more flexible than ever as you can render logs in any format the template ranging from CSV to XHTML.

The variables for each log entry are date, level and message.

Here’s an example of using the zTemplate plugin to create a load of SQL insert statements. Why would you do that instead of inserting directly into the DB ? On a very busy site it can pay to write to a text file and import the logs into the DB when the server is less busy, e.g. during nightly maintenance.

$logger = new \opensourceame\logger( array(
  'plugin'       => 'zTemplate',
  'templateFile' => 'myTemplate.tpl',
  'logFile'      => 'logs/' . date('Ymd_His') . '.sql',
));

And the template file:

 
<!-- BEGIN: logger -->
USE myLogDB;
<!-- BEGIN: log -->
INSERT into myLogTable (date_time, priority, info) values ('{date}', '{level}', '{message}');
<!-- END: log -->
<!-- END: logger -->
Shortlink

Logger and docs updated

I’ve done some updates to my Flexible Logger, including:

  • XML plugin can output date and level info embedded as attributes instead of tags
  • HTML plugin accepts an option cssClasses to specify classes for table, rows and cells
  • improved buffering for all plugins

I have also made the API docs available online using APIGen, my new documentation generator of choice.

Shortlink

Quickly format tables for SugarCRM

If you are hand-coding tables for SugarCRM then you can skip the tedious addition of attributes to table elements by using my sugarTables.js script to quickly format a table in a SugarCRM style. The script requires PrototypeJS.

For example, creating a table like this:

 
<script type='text/javascript' src='/custom/include/javascript/prototype.js'></script>
<script type='text/javascript' src='/custom/include/javascript/sugarTables.js'></script>
 
<table id='myTable'>
	<tr>
		<td>Name:</td>
		<td>Joe Soap</td>
		<td>Age:</td>
		<td>25</td>
	</tr>
	<tr>
		<td>Postcode:</td>
		<td>AB12 3EF</td>
		<td>Country:</td>
		<td>United Kingdom</td>
	</tr>
</table>
 
<script type='text/javascript'>
 
$('myTable').makeSugarDetail();
 
</script>

renders a table looking like this:

The makeSugarList() method works in the same way as makeSugarDetail() except that it formats a table in Sugar’s standard list format.

Both functions return the element so that you can add further formatting inline if desired, as in the example below which makes the table 100% wide after formatting in SugarCRM styling:

$('myTable').makeSugarDetail().setStyle({ width: '100%' });

Download the script here

There are a lot of tables hard coded in SugarCRM. I would dearly like to see Sugar use a standard function like this to format tables rather than manually adding attributes into the original code. There are far too many errors in Sugar’s HTML code, all down to human error, and these could be easily avoided.

Shortlink

CETeams-1.0.7d-p6

I have updated my patched version of CETeams 1.0.7d to version 6 which includes changes to make the module work with Sugar 6.1.6. You can download the zip file of the source below:

CETeams-1.0.7d-p6

Shortlink

Sugar SubPanels and SubQueries

Sugar allows you to specify a where clause in a subpanel This is useful if you want to show only certain records in the subpanel, or to filter the subpanel results, as detailed in my blog entry Filtering Sugar Subpanels.

While it’s a useful feature, it does have a problem – subqueries. If you specify something like this:

"where" = "id in (select id from some_table where field='xyz')"

Sugar will make a mess of your subquery. There’s an easy solution. Sugar handles query rewriting based on the strings “where” or “WHERE”. So, all you need to do is write your “WHERE” in mixed case, like so:

"where" = "id in (select id from some_table WhErE field='xyz')"

In my experience most, if not all, database servers will happily run the query with the where clause written as “WhErE”. Problem solved :-)

Shortlink

Filtering Sugar SubPanels

Sugar doesn’t have a means of filtering subpanels built into it. This is a serious drawback if you have subpanels with a lot of data in them. However, it’s possible to create your own filter with a little work. I’ll explain my solution here.

First, here’s a screenshot of a notes subpanel for a client record on an installation of Sugar that I work on:

The company makes customer advice agents record a contact with every note in a client record, and as the list of notes grows it becomes important to be able to filter them by contact. To do this I create a selector with an onchange parameter that calls Sugar’s showSubPanel() javascript function.

I need to pass the id of the contact to filter by as a parameter in the request. showSubPanel() has a fourth parameter, layout_def_key.

Any value passed to that parameter causes “&layout_def_key=xyz” to be appended to the URL. You can inject any other parameters you like into the URL using this parameter.
In my case I pass the value “&contact_id=abc” which causes the URL to be formatted something like this:

/index.php?sugar_body_only=1&module=myCustomModule…record=abc&layout_def_key=&contact_id=xyz

Now, since I have a contact_id parameter in the request, I can add a where clause to my subpanel in custom/modules/Notes/metadata/subpanels/ForMyCustomModule.php like so:

$subpanel_layout = array(
    'sort_by'        => 'date_modified',
    'sort_order'     => 'desc',
 
    'top_buttons'    => array(
        array('widget_class' => 'SubPanelContactFilter',   'subpanel' => 'notes'),
        array('widget_class' => 'SubPanelTopSelectButton', 'popup_module' => 'Notes'),
    ),
 
    'where' => ( isset($_REQUEST['contact_id']) and ($_REQUEST['contact_id'] != null) ) ? " notes.contact_id='$_REQUEST[contact_id]' " : null,
 
    'list_fields' => array(

To render the contact list I create a widget which is configured in the subpanel layout definitions as a button. It’s the first button in the above example, called SubPanelContactFilter.

The code for the widget is stored in include/generic/SugarWidgets/SugarWidgetSubPanelContactFilter.php and looks something like this:

 

class SugarWidgetSubPanelContactSelector extends SugarWidget
{
	public function display(&$data)
	{
		$contact_id = isset($_REQUEST['contact_id']) ? $_REQUEST['contact_id'] : null;
 
		/*
		 * add your code to fetch your contact list here
		 */
 
		if(empty($GLOBALS['contacts']))
			return false;
 
		// get the subpanel name
		foreach($data['subpanel_definition']->panel_definition['top_buttons'] as $button)
		{
			if($button['widget_class'] == 'SubPanelContactFilter')
				$subpanel_name = $button['subpanel'];
		}
 
		// create the HTML code to return
 
		$html  = "<select id='contactFilter_{$subpanel_name}' onchange='showSubPanel(\"$subpanel_name\", null, true, \"&contact_id=\"+this.value);'>";
		$html .= "<option value='- show all -'></option>";
 
		foreach($GLOBALS['cr_contact_list'] as $c)
		{
			$html .= "<option value='$c[id]'>$c[first_name] $c[last_name]</option>";	
		}
 
		$html .= "</select>"; 
 
		// set the selector to the chosen value
		$html .= "\n<script type='text/javascript'>document.getElementById('contactSelect_{$subpanel_name}').value = '$contact_id';</script>"; 
 
		return $html;
 
	}
}

Now, you can see in my code that I use javascript to set the value of the select to the chosen contact. This is because the showSubPanel() function, when inserting the newly fetched data into the subpanel, prevents any use of the “selected” attribute on a select option from working.

In order to make the javascript execute I also had to modify showSubPanel(), adding a line to cause the script to be executed, like so:

var returnstuff = http_fetch_sync(url+ '&inline=' + inline + '&ajaxSubpanel=true');
request_id++;
got_data(returnstuff, inline);
 
// added to cause javascript to be executed after fetching the subpanel data
SUGAR.util.evalScript(returnstuff.responseText);

That completes the work needed to make a subpanel filter. Here’s the filtered result:

Shortlink

Analyse Sugar Queries

Database performance can have a huge affect on the performance of a web application, especially one as large and complex as SugarCRM. Tuning your database to deal with Sugar’s many queries can have a huge impact on its performance. But how do you find out what exactly Sugar is doing, and which queries are causing a bottleneck?

Sugar comes with an option in the logging section of system settings called “log slow queries”. This option adds an extra entry to the log file when a query takes longer than a certain (configurable) threshold to execute. While this is useful to a point it doesn’t really help an admin to find out what queries are using up the most database time. For one thing some queries, while executing very quickly, are repeated many, many times. When you add these repetitions together they can amount to a surprisingly large amount of time.

I wanted a tool to give me a realistic analysis of the queries running on the various Sugar installations I deal with. Finding nothing available online I “rolled my own” and have decided to make my code available for others to use.

Here’s an example of the analysis from my script:

-----  Query Analysis -----

sorting by:                 combined
log file used:              fetch-cr.log
log entries read:           3126
max queries to display:     200
min time to displayed:      0.01
total queries:              537
unique queries:             203
slowest query:              0.429
slowest combination:        0.5058
average query time:         0.0078
average combined time:      0.0208
total unknown time:         0.0408
total db time:              4.2129

----- Query #1 ran 30 times ----- total: 0.5058  max: 0.429  avg: 0.0169 -----

			SELECT *
			FROM contacts
			LEFT JOIN contacts_cstm ON contacts.id = contacts_cstm.id_c
			WHERE contacts_cstm.contact_type_c = 'Core'
			AND id
			IN (
				SELECT contact_id
				FROM eac_clientrecords_contacts
				WHERE cr_id = '5ead17bf-3c54-1a73-6f8d-4a4d74b9ffcb'
			)

----- Query #2 ran 1 times ----- execution time: 0.1621 -----

select top 11 * from ((select tasks.id ,  tasks.name ,  tasks.status  ,  isnull(contacts.first_name,'') + ' ' + isnull(contacts.last_name,'') contact_name ,  tasks.contact_id ,  '

That was an extract from the log when opening a detail view of an object in a custom module. From the analysis I could see that just over four seconds of the time taken to render the page was taken by database time. From that I was able to change some queries’ syntax to make them run faster, and add indexes and other optimisations to the database where queries couldn’t be changed, eventually cutting the db time down by almost 75%.

You may wonder what “unknown time” means in the statistics. This is db time accounted for by Sugar where no query was output to the log. For example, you’ll see entries like this in your log file:

03/29/11 11:03:55 [10980][-none-][DEBUG] Ignoring this database message:
03/29/11 11:03:55 [10980][-none-][INFO] Query Execution Time:0.021867036819458

The easiest way to get a log extract for a specific action (e.g. fetching a contact list) is as follows:

  1. delete the sugarcrm.log file
  2. perform the action (i.e. open the contact list view) in a browser
  3. rename the log file to something like contact-list.log

You can then run the analysis script against that particular log file using the -f parameter. Note that the script is currently only designed to handle Sugar’s standard log format.

The options available are:

  • Sorting - sort by individual query time, combined time of repeated queries or number of times a query was run
  • Untimed queries - include queries run where Sugar didn’t write a corresponding execution time to the log
  • Thresholds – choose how many queries to display, and the minimum query time

Running the script

Before you begin you will need to set Sugar’s logging level to “debug” so that it logs the details for every query run.

The script needs to be run from a command line, as below:


./analyse_sugar_queries.php -f contact-list.log -s combined -mt 0.001

Download the script using the link below:

analyse_sugar_queries

If you’d like to see any other features added to the script I’d love to hear about them, and any feedback on how the script has helped you analyse your SugarCRM queries :-)

Shortlink

CE Teams 1.0.7d patch 5

I’ve done another update to the CE Teams code to better support Sugar 6.1. This patch fixes an issue with the User Management page showing as a blank and updates the Unified Search code. If you’re using CE Teams with SugarCRM 6.1 this is the patch to use.

Download Here