magento code audit
DESCRIPTION
TRANSCRIPT
![Page 1: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/1.jpg)
Magento Code Audit
Magento Expert Consultant Group Oleksandr Zarichnyi, Vitaliy Stepanenko
![Page 2: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/2.jpg)
• Issues detected in code
• How we conduct code audit
• Value code audit brings to the table
Will talk about
![Page 3: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/3.jpg)
What is code audit?
![Page 4: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/4.jpg)
Projects
Health Check
Upgrade Analysis
Before Launch Check
Crash Investigation
![Page 5: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/5.jpg)
Experience
50+ projects
6670474 LOC
74396 classes
290594 methods
45860 issues
![Page 6: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/6.jpg)
Issues
![Page 7: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/7.jpg)
Issue 1
throw new Exception( "Cannot find product " + $this->getSku() );
![Page 8: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/8.jpg)
throw new Exception( "Cannot find product " . $this->getSku() );
Issue 1
![Page 9: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/9.jpg)
protected function _revertById($id, $amount = 0) { $giftCard = Mage::getModel('giftcard/giftcard') ->load($id); if ($giftCard) { $giftCard->revert($amount) ->unsOrder() ->save(); } return $this; }
Issue 2
![Page 10: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/10.jpg)
Expression is Always True
![Page 11: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/11.jpg)
Issue 2
protected function _revertById($id, $amount = 0) { $giftCard = Mage::getModel('giftcard/giftcard') ->load($id); if ($giftCard->getId()) { $giftCard->revert($amount) ->unsOrder() ->save(); } return $this; }
![Page 12: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/12.jpg)
for ($i = 0; $i < count($data); $i++) { //.. }
Issue 3
![Page 13: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/13.jpg)
Issue 3
$count = count($data); for ($i = 0; $i < $count; $i++) { //.. }
![Page 14: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/14.jpg)
Issue 4
public function getRandomProduct() { $collection = Mage::getModel('catalog/product') ->getCollection() ->addStoreFilter() ->getSelect() ->order('RAND()'); return $collection->getFirstItem(); }
![Page 15: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/15.jpg)
Fetching More Than Necessary
![Page 16: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/16.jpg)
Issue 4
public function getRandomProduct() { $collection = Mage::getModel('catalog/product') ->getCollection() ->addStoreFilter() ->getSelect() ->limit(1) ->order('RAND()'); return $collection->getFirstItem(); }
![Page 17: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/17.jpg)
![Page 18: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/18.jpg)
Code Smell
![Page 19: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/19.jpg)
FIXME
TO DO
HA CK
![Page 20: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/20.jpg)
Axe Effect
![Page 21: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/21.jpg)
cwe.mitre.org
250 internally mined common entries + 200 entries from other sources ECG
• Template for issue description • Catalog of 400 entries
applicable for PHP and Magento code
Describing Issues
![Page 22: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/22.jpg)
Name
Description
Recommendation
Level of Effort
Priority
Relationships
![Page 23: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/23.jpg)
Architecture and Design Implementation
Installation and Upgrade Configuration
Time of Introduction
![Page 24: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/24.jpg)
Impact Accessibility Accountability Adaptability Administrability Affordability Agility Availability Capability Composability Configurability Compatibility Demonstrability Deployability Durability
Executability Extensibility Evolvability Fidelity Flexibility Functionality Integratability Interoperability Interpretability Maintainability Manageability Mobility Modifiability Operability
Performability Portability Practibilty Practicality Predictability Producibility Recoverability Reliability Repeatability Responsibility Reusability Scalability Serviceability Stability
Supportability Suitability Survivability Tailorability Testability Traceability Trainability Transportability Trustability Understandability Upgradability Usability Verifiability Vulnerability
![Page 25: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/25.jpg)
Product Quality Model
![Page 26: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/26.jpg)
Deliverable: Report
![Page 27: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/27.jpg)
![Page 28: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/28.jpg)
Trends • Most popular issues • Issues breakdown by location, impact, time of
introduction • Overall code quality
• Better understanding nature of the issues
![Page 29: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/29.jpg)
How to Survive?
![Page 30: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/30.jpg)
A lot of routine tasks
A lot of data
A lot of formal stuff
![Page 31: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/31.jpg)
• reVu IDE plugin
• Automated code analyzers
• Report generators
• Data refine tools
ECG Toolkit
![Page 33: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/33.jpg)
Code Audit Automation
Vitaliy Stepanenko
![Page 34: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/34.jpg)
![Page 35: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/35.jpg)
Software Audit Tools
1. Static code analyzers 2. Dynamic code analyzers 3. Utilities
![Page 36: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/36.jpg)
![Page 37: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/37.jpg)
Workflow
• Sniffing
• Collecting & merging results
• Exporting data to reVu
• Manual review in reVu
• Generating final report
![Page 38: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/38.jpg)
Code Sniffers
PhpMd (PHP mess detector)
Php_CodeSniffer
![Page 39: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/39.jpg)
How to sniff?
Reflection
Parsing Tokenization
RegExp? Token Lexeme Line
T_OPEN_TAG <?php 1
T_COMMENT /**@var $a bool */ 2
T_VARIABLE $a 3
T_EQUAL = 3
T_LNUMBER 2 3
T_IS_NOT_EQUAL <> 3
T_LNUMBER 1 3
T_SEMICOLON ; 3
<?php /**@var $a bool */ $a = 2 <> 1;
![Page 40: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/40.jpg)
![Page 41: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/41.jpg)
Issues outside PHP code
Xml files (configuration & layout updates)
DB Schema (indexes, non-optimal field types)
Wrong file’s placing & naming
Javascript, CSS & HTML issues
![Page 42: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/42.jpg)
Working on compound sniffers
1. Many different approaches which should be used together
2. Calculations redundancy Tokenize code again and again by each sniffer Typically Magento application have over 8,000 files consisting of code, templates, JavaScript and CSS
Difficulties
![Page 43: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/43.jpg)
![Page 44: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/44.jpg)
Solutions: software graph
1. File system as part of graph
![Page 45: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/45.jpg)
![Page 46: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/46.jpg)
Software graph
1. File system as part of graph
2. PHP Reflection as part of graph (TokenReflection)
![Page 47: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/47.jpg)
![Page 48: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/48.jpg)
Software graph
1. File system as part of graph
2. PHP Reflection as part of graph (TokenReflection)
3. PHP lexical tree inside methods & functions as part of graph (PHP_Parser)
![Page 49: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/49.jpg)
Software graph
1.Back links, circular links (parent class, overridden method)
2.Typed connections, polymorphism
Semantic relations: • Holonymy & meronymy • Hyponymy & Hyperonymy
![Page 50: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/50.jpg)
Node families & extensibility 1. File system 2. PHP • Reflection (classes, methods, namespaces, etc) • PhpDepend (metrics for reflection objects) • Lexical tree (inside php functions) 3. Magento • Directory-based
Magento application, code pools, namespaces, modules • Class-based
models, controllers, blocks, helpers • File-based
Install & upgrade scripts, configuration files, layout updates extends files 4. Other programming languages? 5. Git, SVN? 6. Virtual nodes • Magento functional scopes • Specific code (ex: performing DB Queries)
![Page 51: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/51.jpg)
Software Graph’s API
• Visitor • Direct querying
search methods, fluent interface, state monad • Query language
just syntactic sugar
![Page 52: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/52.jpg)
Software graph: additional benefits
1. Query caching, lazy loading
2. Intelligent node search, traverse algorithms based on relation types
3. Easy way to get path (issue location) File Class Name Method name Line numbers
![Page 53: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/53.jpg)
Query Language Implementation
Parser: Built with Loco, parser combinator for PHP Interpreter: State monad wrapper for graph traverse API + 1. Simple boolean operators 2. Tunneling to native php functions
![Page 54: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/54.jpg)
Examples
![Page 55: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/55.jpg)
Example 1 Find model load in loops \LoopStatement.body\MethodCall[name = “load”]
class Ecg_Sniffs_Performance_LoopModelLoadSniff implements PHP_CodeSniffer_Sniff { public function register() { return array(T_WHILE, T_FOR, T_FOREACH, T_DO); } public function process(PHP_CodeSniffer_File $phpcsFile, $stackPtr) { $tokens = $phpcsFile->getTokens(); $opener = $tokens[$stackPtr]['scope_opener']; $closer = $tokens[$stackPtr]['scope_closer']; for ($ptr = $opener + 1; $ptr < $closer; $ptr++) { $content = $tokens[$ptr]['content']; if ($tokens[$ptr]['code'] === T_STRING && $content == 'load') { $phpcsFile->addError('Model load in loop detected', $ptr, 'ModelLoad', array $content)); } } } }
//*[ name()="node:Stmt_Foreach" or name()="node:Stmt_Do" or name()="node:Stmt_For" or name()="node:Stmt_While" ]//node:Expr_MethodCall/subNode:name[ scalar:string = "load" ]
![Page 56: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/56.jpg)
Example 2
Find all methods in code that has inconsistence between docBlock annotation and really returned value Method [ \DocBlock.returnAnnotation.types as $types, \Statement [ name=“return”, !(expression.returnedType in $types) ] ]
![Page 57: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/57.jpg)
Example 3
Find direct output in models \(MageModel or MageResourceModel)\OutputStatement
![Page 58: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/58.jpg)
Rule Examples 1. Perhaps DB query not inside resource model or install/upgrade script is an issue
2. DB query inside block and controller definitely is an issue
Next concept: confidence
Perhaps? Definitely?
Two types of confidence 1. Confidence based on accuracy of sniffs
Any rules have exceptions
2. Confidence based on accuracy of observations Used technologies are not ideal
![Page 59: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/59.jpg)
Code Bases
1. Target codebase Concrete module, local code pool
2. Auxiliary codebase PEAR libs, whole Magento application
Example: Analyzed class inside target code base, parent class inside auxiliary codebase. We search for copy-pasted code in overridden methods without parent’s method call.
![Page 61: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/61.jpg)
References
https://github.com/magento-ecg/coding-standard – ECG CodeSniffer coding standard
http://cwe.mitre.org – Common Weakness Enumeration
https://github.com/syllant/idea-plugin-revu – reVu code review plugin
https://github.com/nikic/PHP-Parser – PHP Parser
http://stackoverflow.com/questions/1732348/regex-match-open-tags-except-xhtml-self-
contained-tags – Epic answer about parsing HTML with regular expressions
http://phpmd.org/ – PHP Mess Detector
https://github.com/Andrewsville/PHP-Token-Reflection – PHP Token Reflection
![Page 62: Magento code audit](https://reader031.vdocuments.site/reader031/viewer/2022020115/5481d2dbb079596a0c8b4609/html5/thumbnails/62.jpg)
Questions