Improving Drupal Performance

Drupal Surrey Group, June 2013

By Vladimir Ilic

Agenda

About me Get Drupal 7 working faster Optimize code in order to get proper

responses Use cache (memcache, APC cache, entity

cache, varnish) Scale Drupal horizontally in order to

balance load

About me

name: Vladimir Ilic

email: burger.boy.daddy@gmail.com

twitter: @burgerboydaddy

http://burgerboydaddy.com

Why?

When it comes to attracting and keeping site visitors, the research is clear - fast sites win out over slower ones.

Users simply don’t have much patience with sites that take too long to load and that impatience isn't measured in seconds, but milliseconds.

57% of mobile shoppers will wait 3 seconds or less before abandoning the site!

Start point

Before, before start Think about site performance before you start building site.

But if you already have a site up and running... First measure how fast is site right now

https://developers.google.com/web-toolkit/speedtracer/ Yslow Load test (blitz.io)

If it fast, responsive, and scale good under load do not touch

If it is slow time to change Architecture Software Hardware (no matter how easy it looks like, leave HW as last

one) Oh, yes, there is MySQL; and that is most important point of your

site (from every point of view including speed), but improving database performance is a part of all 3 points mentioned above.

Site Architecture

Drupal developers have tendency to add every module that exist on the planet, and you (as great Drupal developer) may want to install them all, but don’t. And many developers keep all of them enabled

How many times you noticed that some site has 150 – 200+ modules enabled, and only need 10-15 of them.

And every one (module) will add some execution time.

Bad modules

Drupal core ships with some great modules but it also ships with some nasty ones. Here are 3 of the worst: Database logging (dblog) (use syslog instead) Statistics (use Google Analytics) PHP Filter (put all of your code in modules!!!)

Tips #1

Do not test module on a live production site. For “how this module works” use some

dummy development site created for those purpose only.

Properly disable / uninstall module disable it Go to “/admin/modules/uninstall “and uninstall it

Life-saver. Mistakenly deleted a module from the directory without disabling it first will kill site performance SELECT name, filename FROM system WHERE type = 'module' AND status = 1

ORDER BY filename And clean-up any modules that are still enabled but missing from the file

system.

Plan ahead

Think about performance before starting to code

Think about Modules that will be used by site (again: test

somewhere else) Is UI heavy loaded? Do we need to connect to external data

sources (web services, external databases, etc)?

Can we use CDN (content delivery network)? Add expire headers (apache mod_expires)

MySQL

Drupal sites live or die by their database. Is database optimized for speed

MySQL version, InnoDB vs MyISAM, (phpMyAdmin still use as

default MyISAM and developers do not care). Can we tweak MySQL settings or they are locked

by ISP Can we install additional cache on server (do I

have full access to server or it is on shared hosting environment)

Make sure the tables are properly indexed for faster searching.

Tips #2

Custom tables They are ok but DO NOT FORGET INDEXES!!!

Use slow queries log (great to find system bottlenecks)

MyISAM vs. InnoDBMy ISAM InnoDB

Required full text Search Yes Require Transactions Yesfrequent select queries Yes frequent insert, update, delete YesRow Locking (multi processing on single table) YesRelational base design Yes

UI Tweeks

Tweak Your Site Images No, you do not want to allow users to download 2-3MB

image that will be scaled to 100x150px inside user browser.

Resize your images and compress them prior to uploading or use Image resize filter to do it.

Image resize filter is a smart piece of code that will automatically resize all the images on your website (not the dimensions), and makes Drupal performance much faster for your visitors.

Use Image Sprites (spritecow.com)

Software…

What I build I can control. Great one from Chris Baus

“Talented, but inexperienced, developers tend to use every trick in their bag to show off their knowledge and capability. I've seen the harm that can be done when someone heads off into the wilds of a code base with such a mindset. It can be toxic, and it is often in the spirit of impressing other developers. Experienced developers make the difficult seem easy. New developers often make the routine look hard.” http://baus.net/you-cant-impress-developers/

Code…

Be sure to check are there any PHP errors. Avoid PHP errors that occur within nested foreach

loops at all costs. Once when site is in Production clean your

watchdog logs calls. Every watchdog log consumes CPU resources on the web and database server (if you decided to keep logs in db please don’t). It also increases load time significantly.

But be sure to check your logs for warnings and errors. They are best source of info what’s wrong.

Team work is good if…

Code review is used it to improve code execution time, not how nice it looks. Leave code formatting to automated tools. Use your review time to improve code functionality

and speed. Parallel programming is ok if both heads are

used to improve code. Use recommended Drupal coding practices

Programming best practices (https://drupal.org/node/287350)

Coding Standards (https://drupal.org/coding-standards)

Using Views?

When optimizing Views many of the same rules apply as optimizing database queries.

In the Views interface when you “Preview” a view it will show you the query it’s generating and from there it may be clearer what is going on under the hood.

Be sure to optimize your views queries

Views – bad example

Example:SELECT node.nid AS nid, product.upcs AS product_upcs, product.image_url AS

product_image_url, node.title AS node_title, product_price.points_only AS product_price_points_only, product_price.point_price_dollars AS product_price_point_price_dollars, product_price.point_price_points AS product_price_point_price_points, product.top_tag_line AS product_top_tag_line, product.brand_name AS product_brand_name, node.created AS node_created

FROM {node} nodeLEFT JOIN {product} product ON node.nid = product.nidLEFT JOIN {product_price} product_price ON node.nid = product_price.nid

WHERE (( (node.status = '1') AND (node.type IN ('product')) AND

(product.end_date > 1371103484+0) AND (product.start_date <= 1371103484+0) ))

ORDER BY product_top_tag_line ASC, node_created DESCLIMIT 8 OFFSET 0

Above query cannot be cached because where clause is bonded with current time. Do we really need that?- In this case not; For developer was easier to add current date/time but in real life we changed product start/end date only once a day!!!

Views - improvements

Look at ways not to use “distinct” and “count” in your queries.

Try a few different settings within Views, and a few different versions of the queries to see which ones load faster, you may be able to get much better performance by only slightly compromising on functionality.

Go to admin/structure/views/settings

Views – improvement cont…

Check view caching (select as long as possible cache time).

Use Views lite pager (drupal.org/project/views_litepager/).

You do not need all items/articles loaded at the same time. 10-20 is sufficient in most cases. Example: Product manger asked developer to

remove pager and show all catalogue items during load (she hates to click “Show All” every time when she need to review all catalogue items). Young developer did that (to impress business user how he can do that in a second). Not long time after that clients started to complain about load time…

Cache

No brainer: Cache your cache!!! Always enable cache on site (at least

default one for anonymous users). Go to Configuration, Performance and

enable your cache. Once when you are there setup

“Minimum cache life time”. Compress and aggregate your JS, CSS

files. If you can leave compression to Apache server

(deflate_module); it will do that faster.

memcached

Drupal has a fantastic hook-able caching system, where any module can write to a standard cache table, or create a cache table, then use a specific API to write to these cache tables.

When using these cache tables it can save large complex PHP tasks or MySQL queries, but it can also create more slow queries for reading and writing the cache.

Memcache relieves that problem by storing all of these cache tables in memory. For many sites these reduces load on the server and increases the performance of the site.

Memcached has three components, the Memcached software, a PHP extension and the Drupal Memcache module that work together to provide in-

memory storage of database calls or rendered pages which makes it very fast.

APC

APC (Alternative PHP Cache) is a PHP OP code cache.

It is a very quick win when working with PHP and can offer a great performance boost when using Drupal. It is very much a “set it and forget it” type of application which can just be installed, enabled and left to do it’s thing.

Many Drupal specific hosting companies will already have APC setup and running so you may even be using it without noticing.

APC 2

Install Drupal APC module (drupal.org/projects/apc)

Use APC for caches that do not change often and will not grow too big to avoid fragmentation. The default setting of APC will allow you to store 32 MB

for the opcode cache and the user cache combined. Make sure you tweak this according to your website's

needs. An example configuration could be to cache 'cache'

and 'cache_bootstrap' in APC; 'cache_field' and 'cache_menu' in Memcached and store 'cache_filter' in the database

Xcache and eaccelerator are other options

Varnish

When you have a lot of anonymous users reverse proxy cache can save you a lot of server load.

Varnish is one of the more popular solutions within the Drupal world.

Varnish sits in front of your web server application, for example Apache, nginx or lighttpd, and can run on the same server or a remote server.

It is often run on a load balancer in front of multiple web servers.

Varnish will cache pages for anonymous users, for as long as the “max_age” header is set.

Varnish 2

Varnish can be quiet complex to setup, the there are many Drupal focused tutorials.

In addition to the Varnish Cache software, you’ll need the Drupal module (drupal.org/project/varnish). According to the module page, Varnish will serve pages at a much

faster rate than Apache - close to 3,000 page views per second! It’s advised to configure it to only bypass the cache for users

with a cookie starting with “SESS” as these are given to authenticated Drupal users, but any module that sets “$_SESSION” in it’s code will also set one of these cookies in Drupal, which will cause Varnish to be bypassed, and extra load to be added to the web server.

Also note that when a cached page is served from Varnish, no PHP code will get executed within Drupal, therefore things such as mobile detection, or geoip detection will not function.

MongoDB

There are other ways of speeding up your site besides caching. MongoDB is a ‘NoSQL’ type of database that can be used for your Drupal site. Yes, this means for some tables we will replace MySQL with MongoDB.

MongoDB avoids resource hogging JOIN statements by using document based records.

Another thing that really sets MongoDB apart is that the database is stored in memory, so writes are very fast. MongoDB does do occasional writes back to disk, but such traffic is greatly reduced. One criticism of keeping the database in memory is that some data would be lost in the event of a server crash. With the latest version, however, this issue can be solved with a simple configuration tweak.

Oh, and to get MongoDB up and running, you'll need the Drupal module.

CDN (Content Delivery Network)

Content Delivery Networks are like alternative servers for your website that are spread all over the world- and serve pages to your visitors from the closest network to them.

CDNs are a segment of Cloud computing and provide lighting fast speed to websites.

A CDN is used to distribute static assets such as images, documents, CSS and JavaScript across many locations, so can be useful if you target an international audience.

If you only target a more local audience then serving your static assets from Varnish may actually work out faster.

Easy to setup using CDN module (drupal.org/project/cdn) and Amazon S3 as backend.

Hosting

Be Realistic About Your Hosting

If things still don’t work then just stop where you are and think of increasing your hosting configuration. Lack of server resources is also one of the reasons for a slow website.

Shared or conventional hosting environment is known to be a performance barrier. If you feel that even higher resource allocation is not doing any

good, then you must choose some other decent hosting solution.

Cloud hosting environment best suits any type of Website. Its limitless resources and metered payment format suits the resource needs of all kinds of businesses.

Tips #x

Extend mod_expires settings (make sure its on) in Drupal .htaccess

.htconfig move to httpd.conf eliminates Apache parse and search on every load

Search is resource intensive Consider moving to Apache Solr Disable “Update Manager” module in Production.

You do not want Drupal to waste time during every cron run to consult drupal.org about modules updates.

Disable developer modules (Views UI is included in that request)

Tips #xx

Disable automatic run of Cron job. By default it runs every 3h, and it is attached to every

page hit (check for passed time). Set to never and run your own cron job

Increase garbage collector run frequency for busy site inside settings.php Session.gc_maxlifetime Session.cache_expire

Cache warming By using cache warmer module (https://drupal.org/project/cache_warmer)

or Manually using:

#!/bin/bashwget -q -O - http://your-domain.com/sitemap.xml | egrep -o "http://your-domain.com[^<]+" | wget -q -i - -O -

Thanks for your patience

Any Question?