magento indexes
DESCRIPTION
Presentation from Magento MeetUp on 17th June in AmsterdamTRANSCRIPT
Magento Indexers
Ivan ChepurnyiMagento Trainer / Lead Developer
Ivan Chepurnyi Magento Developers Meetup
Agenda
• Overview of Indexes Functionality• Creation of own indexes
Ivan Chepurnyi Magento Developers Meetup
Let Imagine…
… that Magento doesn’t have indexes:
• The prices in product list are calculated on the fly depending on catalog rules, tier prices for customer groups
• Stock availability for configurable and bundle products can be calculated only after loading the product collection
• Layered navigation data is build in real-time for product attributes information
• Anchor categories recursively collects subcategories for filtering product list
Ivan Chepurnyi Magento Developers Meetup
It’s all about performance…
The main goal is minimizing amount of operations to display
products to a customer
Ivan Chepurnyi Magento Developers Meetup
Definitions
• Indexed DataAggregated data for entity representation on the frontend lists.
• Indexer
Generates index data on event or manual by process.• Index Event
The moment when entity or related to it information is changed and that affects its index data.
• Index Process
Wrapper for indexer and contains information about its mode and status
• Main Controller
Forwards events to Index Process
Ivan Chepurnyi Magento Developers Meetup
Index Workflow
EventEventEvents
Process
Main Controller
Indexer
ManualInvoke
Indexed Data
Ivan Chepurnyi Magento Developers Meetup
Event Types
• Save
When indexed entity or related to it information was changed
• Delete
When indexed entity or related to it one was deleted
• Mass UpdateWhen batch of entities was updated. (Update Attributes on Product Grid)
Ivan Chepurnyi Magento Developers Meetup
Observed Entities
Indexed Entities– Product– Product Inventory– Category– Tag
Entities Scope– Customer Group– Website– Store Group– Store View
Ivan Chepurnyi Magento Developers Meetup
Index Process
Available Statuses
• Pending
Indicates that indexer is up to date
• Running
Index currently in process of full rebuilding index data.
• Require ReindexStatus for notifying admin user, that index is not up to date and should be rebuild.
Ivan Chepurnyi Magento Developers Meetup
Index Process
Indexing Modes• Real-time
• Manual
Event
EventRequire Reindex
Update Index Data
Ivan Chepurnyi Magento Developers Meetup
Indexer Flow
Match Event
Register Event Data
Reindex Data
Main C
ontroller
Index Process
Ivan Chepurnyi Magento Developers Meetup
Mage_Index Module
Main Controller
Process
Indexer Base
Mage_Index_Model_Indexer
Mage_Index_Model_Process
Mage_Index_Model_Indexer_Abstract
Ivan Chepurnyi Magento Developers Meetup
Catalog Module
Index Module
Indexers Modularity
Mage_Index_Model_Indexer_Abstract
Mage_Catalog_Model_Product_Indexer_Eav
Mage_Catalog_Model_Product_Indexer_Flat
Mage_Catalog_Model_Product_Indexer_Price
Inventory Module
Mage_CatalogIndex_Model_Indexer_Stock
…
Ivan Chepurnyi Magento Developers Meetup
ModelMage_Index_Model_Indexer_Abstract
Indexer Structure
Resource ModelMage_Index_Model_Mysql4_Abstract
Matches event data and runs appropriate method in resource model for re-indexing
Works directly with database for generation of the indexed data. Usually all the data operated via MySQL queries.
Ivan Chepurnyi Magento Developers Meetup
What can you use?
Mage_Index_Model_Indexer
getProcessByCode($indexerCode)
getProcessCollection()
processEntityAction($entity, $entityType, $eventType)
Mage_Index_Model_Process
reindexAll()
reindexEverything()
setMode($mode)
Ivan Chepurnyi Magento Developers Meetup
What you shouldn’t do…
• Invoke reindexAll method from index model/resource model, because it is better to let admin user know when the index was rebuild.
• Process entity events directly with indexer, instead of passing data through the main controller. You never know which index may depend on this event.
Ivan Chepurnyi Magento Developers Meetup
Creating own indexer
• Defining indexer in configuration• Designing index data table• Implementing model • Implementing resource model• Applying index on the frontend
Ivan Chepurnyi Magento Developers Meetup
Featured Products
There is easier way to create featured products functionality, but it is a simple example on what should be done for creation own indexer.
Ivan Chepurnyi Magento Developers Meetup
Defining index in configuration
<config>
<!-- …. module configurtaions -->
<global>
<!-- …. module configurtaions -->
<index>
<indexer>
<featured_products>
<model>your_module/indexer_featured</model>
</featured_products>
</indexer>
</index>
</global>
</config>
etc/config.xml
Indexer Model
Indexer Code
Ivan Chepurnyi Magento Developers Meetup
Designing index data table
• Adding new attribute to catalog product entity called is_featured
• Creating table that will contain product ids of products that are marked as featured products.
Ivan Chepurnyi Magento Developers Meetup
Designing index data table
$this->addAttribute('catalog_product', 'is_featured', array(
'type' => 'int',
'label' => 'Is featured',
'input' => 'select',
'source' => 'eav/entity_attribute_source_boolean',
'user_defined' => false,
'required' => false
));
Setup Script
Attribute Code
Yes/No Dropdown
Ivan Chepurnyi Magento Developers Meetup
Designing index data table
$table = new Varien_Db_Ddl_Table();
$table->setName($this->getTable(‘module/featured'));
$table->addColumn('product_id', Varien_Db_Ddl_Table::TYPE_INTEGER, 11, array(
'unsigned' => true,
'nullable' => false,
'primary' => true
));
$this->getConnection()->createTable($table);
Setup Script Table Alias
Table Column
Ivan Chepurnyi Magento Developers Meetup
Designing index data table
$table = new Varien_Db_Ddl_Table();
$table->setName($this->getTable(‘module/featured'));
$table->addColumn('product_id', Varien_Db_Ddl_Table::TYPE_INTEGER, 11, array(
'unsigned' => true,
'nullable' => false,
'primary' => true
));
$this->getConnection()->createTable($table);
Setup Script Table Alias
Table Column
Ivan Chepurnyi Magento Developers Meetup
Implementing Model
class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract
{
protected $_matchedEntities = array(
Mage_Catalog_Model_Product::ENTITY => array(
Mage_Index_Model_Event::TYPE_SAVE,
Mage_Index_Model_Event::TYPE_MASS_ACTION
)
);
}
Defining Matching Events
Entity Type
Event TypeEvent Types
Ivan Chepurnyi Magento Developers Meetup
Implementing Model
class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract
{
// … other code
protected function _construct()
{
$this->_init(‘your_module/indexer_featured');
}
}
Defining Indexer Resource Model
Resource model
Ivan Chepurnyi Magento Developers Meetup
Implementing Model
class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract
{
// … other code
public function getName()
{
return Mage::helper(‘your_module')->__('Featured Product');
}
public function getDescription()
{
return Mage::helper(‘‘your_module')->__('Indexes something');
}
}
Defining Indexer Information
Indexer Name in the admin
Indexer Description in the admin
Ivan Chepurnyi Magento Developers Meetup
Implementing Model
class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract
{
// … other code
protected function _registerEvent(Mage_Index_Model_Event $event)
{
/* @var $entity Mage_Catalog_Model_Product */
$entity = $event->getDataObject();
if ($entity->dataHasChangedFor('is_featured')) {
$event->setData('product_id', $entity->getId());
} elseif ($entity->getAttributesData()) {
$attributeData = $entity->getAttributesData();
if (isset($attributeData['is_featured'])) {
$event->setData('product_ids', $entity->getProductIds());
}
}
}
}
Register Event for Processing
Product Save Registering
Mass Action Registering
Ivan Chepurnyi Magento Developers Meetup
Implementing Model
class Your_Module_Model_Indexer_Featured extends Mage_Index_Model_Indexer_Abstract
{
// … other code
protected function _processEvent(Mage_Index_Model_Event $event)
{
if ($event->getData('product_id') || $event->getData('product_ids')) {
$this->callEventHandler($event);
}
}
}
Processing Event
Calling processor in resource model
Entity Type Event Type
catalogProductSave($event)
catalogProductMassAction($event)
Ivan Chepurnyi Magento Developers Meetup
Implementing Resource Model
class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract
{
protected function _construct()
{
$this->_setResource(‘your_module');
}
}
Define resource connection
Your module resource prefix
Ivan Chepurnyi Magento Developers Meetup
Implementing Resource Model
class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract
{
// … other code
protected function _reindexEntity($productId = null)
{
$select = $this->_getReadAdapter()->select();
/* @var $attribute Mage_Catalog_Model_Resource_Eav_Attribute */
$attribute = Mage::getSingleton('eav/config')
->getAttribute('catalog_product', 'is_featured');
$select->from($attribute->getBackendTable(), 'entity_id')
->where('value = ?', 1)
->where('attribute_id = ?', $attribute->getId());
Indexing Method
Retrieving only featured product ids
Ivan Chepurnyi Magento Developers Meetup
Implementing Resource Model
if ($productId !== null) {
if (!is_array($productId)) {
$productId = array($productId);
}
$select->where('entity_id IN(?)', $productId);
$this->_getWriteAdapter()->delete(
$this->getTable(‘your_module/featured'),
array(
'product_id IN(?)' => $productId
)
);
} else {
$this->_getWriteAdapter()->truncate($this->getTable(‘your_module/featured'));
}
Indexing MethodIf it is partial re-index, then delete
only related indexed data
Otherwise clear all indexed data
Ivan Chepurnyi Magento Developers Meetup
Implementing Resource Model
$sqlStatement = $select->insertIgnoreFromSelect(
$this->getTable(‘your_module/featured'),
array('product_id')
);
$this->_getWriteAdapter()->query($sqlStatement);
}
}
Indexing MethodFulfill index data from select we
created before
Ivan Chepurnyi Magento Developers Meetup
Implementing Resource Model
class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract
{
// … other code
public function reindexAll()
{
$this->_reindexEntity();
}
}
Handling Events
Full index re-build
Ivan Chepurnyi Magento Developers Meetup
Implementing Resource Model
class Your_Module_Model_Mysql4_Indexer_Featured extends Mage_Index_Model_Mysql4_Abstract
{
// … other code
public function catalogProductSave($event)
{
$this->_reindexEntity($event->getData('product_id'));
}
public function catalogProductMassAction($event)
{
$this->_reindexEntity($event->getData('product_ids'));
}
}
Reindexing Events
Single Save Product Event
Mass Save Product Event
Ivan Chepurnyi Magento Developers Meetup
Applying Index for the frontend
• Observing and event catalog_product_collection_apply_limitations_after– Joining index table to product collection select– Create sub-select filter for collection
Ivan Chepurnyi Magento Developers Meetup
Liked it?
Checkout our advanced
training programs:http://www.ecomdev.org/magento-development-training-programs/advanced
Follow our blog posts:http://www.ecomdev.org/blog
Questions?