fixing magento core for better performance - ivan chepurnyi

Fixing Magento Core for Better Performance

Ivan ChepurnyiCo Owner / Trainer

EcomDev BV

Ivan Chepurnyi Meet Magento

About me

• I am Ukrainian

• Technical Consultant, Co-owner at EcomDev B.V.

• Was one of the first five developers in original Magento core team

• Providing trainings and coaching Magento Developers in Europe

• Main areas of my expertise:– System Architecture – Performance Optimization– Test Driven Development– Complex Customizations


Regular Store Bottlenecks

• Overall Problems

• Category Pages

• Product Pages

• Checkout Pages


Overall Problems


Problem #1

Non optimized layout handles usage

Magento adds on every category and product page a layout handle with its entity id, so you have layout cache unique for

each product or category!


Solution

Add an observer to controller_action_layout_load_before,check loaded handles and remove items that match starts with

CATEGORY_ and PRODUCT_, except PRODUCT_TYPE.


Solution

public function optimizeLayout($observer)

{

$update = $observer->getLayout()->getUpdate();

foreach ($update->getHandles() as $handle) {

if (strpos($handle, 'CATEGORY_') === 0

|| (strpos($handle, 'PRODUCT_') === 0

&& strpos($handle, 'PRODUCT_TYPE_') === false)) {

$update->removeHandle($handle);

}

}

}

Example of Observer


Drawback

Widgets based on Category Id and Product Id handles will not work. Hovewer you always have for them “Custom Layout

Update XML” attribute on entity level.


Problem #2

No cache for CMS Blocks

Magento doesn’t cache them by default. On average projects you have up to 10-20 CMS blocks that can be edited by

customer on every page.


Solution

Add an observer to core_block_abstract_to_html_before,and specify required cache information for the block, like cache

key (that is a block identifier) and cache tags (Mage_Cms_Model_Block::CACHE_TAG).


Solution

public function optimizeCmsBlocksCache($observer)

{

$block = $observer->getBlock();

if ($block instanceof Mage_Cms_Block_Widget_Block

|| $block instanceof Mage_Cms_Block_Block) {

$cacheKeyData = array(

Mage_Cms_Model_Block::CACHE_TAG,

$block->getBlockId(),

Mage::app()->getStore()->getId()

);

$block->setCacheKey(implode('_', $cacheKeyData));

$block->setCacheTags(

array(Mage_Cms_Model_Block::CACHE_TAG)

);

$block->setCacheLifetime(false);

}

}

Example of Observer


Problem #3

Magento top menu

Since CE 1.7 version of Magento, they moved menu to another block, but forget to add caching to it. So now each page load

spends from 100-300mson building top menu!

And the more categories you have, the more time is spent!


Solution

1. Add a cache tags via observer similar to optimization of CMS blocks. The difference only, that you will need to add category cache tag instead: Mage_Catalog_Model_Category::CACHE_TAG

2. Rewrite Mage_Page_Block_Html_Topmenu to add category id into the menu node class name. Yes, there is no event for this :(

3. Add JS on the page that will use current category id for highlighting top menu item.


Solution

public function optimizeNavigationCache($observer){ $block = $observer; if ($block instanceof Mage_Page_Block_Html_Topmenu) { $cacheKeyData = array( Mage_Catalog_Model_Category::CACHE_TAG, 'NAVIGATION', Mage::app()->getStore()->getCode() );

$block->setCacheKey(implode('_', $cacheKeyData)); $block->setCacheTags( array(Mage_Catalog_Model_Category::CACHE_TAG) ); $block->setCacheLifetime(false); }}

Example of Observer


Solution

protected function _getMenuItemClasses(Varien_Data_Tree_Node $item)

{

$classes = parent::_getMenuItemClasses($item);

// Will be something like category-node-#id

$classes[] = $item->getId();

return $classes;

}

Overridden Block Method


Solution

JS implementation is up to you.

I believe you can do that yourself :)


Problem #4

Magento Visitor LoggingMage_Log and Mage_Report modules perform a write

operation on every page view to own tables. After few monthes it becomes very slow, since size of the tables increasing

dramatically.


Solution

1. Remove observers of Mage_Log and Mage_Reports module by config.xml– Events for Mage_Log: http://bit.ly/magelog– Events for Mage_Reports: http://bit.ly/magereports

It will remove observers by using of observer type disabled and repeating the observer XML path.


Drawback

You will not be able to use native Magento reports for visitors activity. Hovewer it they are irrelevant, since most of merchants

don’t use it, since they are using other analytics software.


Category Pages & Product Pages


Problem #1

Layered NavigationFor each attribute you have in Magento and that is marked as filtrable, it will make a call to getAllOptions() of attribute source

model. Even if there is no filter results for it, it will invoke attribute option collection load.

For merchants who have a lot of attributes, it is a huge performance bottleneck.


Solution

• Remove collection usage usage for each item by rewriting Mage_Eav_Model_Entity_Attribute_Source_Table class.

• Implement a static method, that will load values for all attributes at once. If you will not use objects, it won’t take too much memory.

• Override getAllOptions() method to use your static method.

• You can find class by this url: http://bit.ly/mageoption


What it does?

// This one will use collection getData() method

// to retrieve array items instead of collection of objects.

Mage::getResourceModel('eav/entity_attribute_option_collection')

->setPositionOrder('asc')

->setStoreFilter($storeId)

->getData();

On the first call to getAllOptions() it will preload all attribute values for current store in array format.


Problem #2

Dropdown Options on Product PageThe problem is the same as with layered navigation, but not

that much visible if you don’t have too much dropdown attributes to show. Product getOptionText() uses the same

getAllOptions() method call.

This one is automatically fixed by fixing previous one.


What it does?

$key = self::_getCombinedKey($storeId, $value, 'store');

if (isset(self::$_preloadedOptionHash[$key])) {

return self::$_preloadedOptionHash[$key];

}

return false;

On the first call to getOptionText() it will preload all attribute option_id to value array for current store.


Problem #3

Configurable Products

Magento is not using flat version of products for configurable products children retrieval. So every configurable product page is a bottleneck, especially for fashion retailers.


Solution

1. Rewrite the class with this long name: Mage_Catalog_Model_Resource_Product_Type_Configurable_Product_Collection

2. Overriden isEnabledFlat() method that should return the real information about flat availability.

3. Make sure that attributes, that your customer is using, are included into the flat version of catalog. Mark them as “used_in_product_listing”.

4. Fix Magento bug with filters by attribute id


Solution

public function isEnabledFlat()

{

return Mage_Catalog_Model_Resource_Product_Collection::isEnabledFlat();

}

public function addAttributeToFilter($attribute, $condition = null, $joinType = 'inner')

{

if ($this->isEnabledFlat() && is_numeric($attribute)) {

$attribute = $this->getEntity()

->getAttribute($attribute)->getAttributeCode();

}

return parent::addAttributeToFilter($attribute, $condition, $joinType);

}

Overriden Collection


Checkout Pages


Problem #1

Catalog Price RulesEach time when Magento calls collectTotals() method on

quote object, it walks though all items in the quote and invoked getFinalPrice() method on your product. This method

dispatches catalog_product_get_final_price event, that is observed by Mage_CatalogRule module.


Solution

1. Rewrite Mage_CatalogRule_Model_Observer class and create a new method that will preload rule prices for quote.

2. Define $_preloadedPrices property to cache results per quote object.

3. Add an observer in configuration for sales_quote_collect_totals_before event, for original core class alias, but with the method you’ve created.

Full observer class can be found at this url: http://bit.ly/magerule


Solution

public function beforeCollectTotals(Varien_Event_Observer $observer)

{

// … Omited retrieval of product ids and customer group with website

$cacheKey = spl_object_hash($quote);

if (!isset($this->_preloadedPrices[$cacheKey])) {

$this->_preloadedPrices[$cacheKey] = Mage::getResourceSingleton('catalogrule/rule')

->getRulePrices($date, $websiteId, $groupId, $productIds);

}

foreach ($this->_preloadedPrices[$cacheKey] as $productId => $price) {

$key = implode('|', array($date, $websiteId, $groupId, $productId));

$this->_rulePrices[$key] = $price;

}

}

Created method code


There are more issues…

But it is too much for one presentation :)

Let’s better talk about high loaded projects!


Are you using Varnish for your projects?


Varnish Issues

Common issues caused by developers, when they use Varnish on the project

• Developers usually hide poor code quality behind front cache server

• Doing backend callbacks for functionality that can be fully done by modern browser in JavaScript.

• Caching static files by Varnish


ESI include is an evil for Magento

Only when the content for ESI include can be cached by Varnish. It doesn’t make to make ESI includes for shopping

cart, compare and recently viewed reports.


Goog ESI includes

• Magento Top Menu NavigationYou don’t need to clear all varnish cached pages, if your customer updated the name of category or re-arranged the position of items.

• CMS Static BlocksIf you have a static that is used almost on every page, you don’t need to clear page cache, if its content changed.


AJAX Callbacks Issue

AJAX call should be done only when needed.Do not perform a bunch of calls just because the data should be loaded from the server. It is always possible to decrease

amount of calls by using your frontend skills.


Use Cookies & HTML5!

• You can always set a custom cookie in Magento when customer:

– Adds a product to a cart

– Logs in or logs out

– Adds a product to a compare list

• You always can store up to 2Mb of data into sessionStorage of your visitor browser! Only IE7 doesn’t support it.


How it can help you?

• You can decrease amount of AJAX calls to the number of real user actions.– Customer adds product to cart, you make an ajax call to

update session storage– Customer logs in, you make and ajax call to update the it

again– Customer adds product to wishlist or compare products

list, you update a session storage.

So in the end it should be1 action – 1 AJAX call,

and NOT 1 page view – 1 AJAX call!


Recently Viewed Products

For recently viewed products, you even don’t need to make any AJAX calls

– Render a hidden block on the product page

– When customer views a product, add this hidden block to session storage

– On any other page, just render data by JavaScript!


Conclusion

Be smart and use Varnish correctly!


OpenSource Roadmap 2014

1. Finalize EcomDev_Varnish module and make it open source;

2. EcomDev_PHPUnit refactoring for API based fixtures;

3. Working on EcomDev_Index module, to provide alternative of standard indexation mechanism in Magento:– Flat Indexers (failover indexation)– UrlRewrites (full refactor of existing module– Layered Navigation (Sphinx)– Better Search (Sphinx)


Thank You!

Questions?

fixing magento core for better performance - ivan chepurnyi

Technology