Magento on Steroids – Best practice for highest performance

Stefan Wieczorek —  September 12, 2012 — 11 Comments

Magento on Steroids - Best practice for highest performance

There are plenty of blog posts with tips how to increase the performance of a magento store this shows us how important the speed of a magento store is.

As you may heard google is obsessed with speed, speed is one important factor for their search ranking algorithms. Site speed reflects how fast the website (magento shop) responds to web requests.

The speed of a site is not only important for the shop owners, it’s important for all internet user. Faster sites means happy user, happy customers and more sales. If a magento shop responds slowly, customers leave the website and maybe buy products at your competitor.

In my daily work over some years with magento performance issues i want to share my best practices.

In order to get the best performance we have to care about the server with it’s configuration and magento, our shop software.

A) The Server environment – the base for highest performance

  1. Get a dedicated server

    Advantages
    – Whole server power for you
    – You are very flexible in terms of new versions for webserver, database, php …
    – There are not other customers which run “bad” scripts which slow down the shop
    – You don’t share the same ip, very important for SEO.
    If one customer makes a “bad business”, the ip can be blocked by google which affects your search ranking.
    My recommendation:
    I am a big fan of the Amazon EC2 cloud which gives you everything you need (flexibility, scalability, security).
    I installed, configured and optimized more than 100 magento shops around the world, works perfect.

  2. Split database and webserver (if possible)

    A webserver and a database server have different requirements. A database server needs fast hard disks (e.g. SSD) and much memory and not that much CPU. A webserver needs more CPU and less memory.

  3. Use a separate Backend Server (Multi Server Environment)

    Run all cronjobs on this server. You can also handle all admin users there.

  4. Use newest MySQL Version
  5. MySQL Configuration

    Proper MySQL configuration is one of the most important aspects in terms of performance. Optimizing the MySQL configuration can provide up to 65% performance improvement. MySQL by default is configured to use far fewer resources than the average hardware can accommodate. InnoDB, the primary table storage engine type can use the in-memory buffer pool to cache table indexes and data. Less disk I/O is needed to get data from hard drives when the value of the in-memory buffer pool is set higher. A general recommendation is to set this parameter up to 80% of the available RAM for a dedicated database server. If you are running webserver and database server on one server it’s recommended to split the entire memory pool into two parts.

    Setting for the key parameter “innodb_buffer_pool_size”

    Server Type innodb_buffer_pool_size
    Combined webserver and database server, 6 GB RAM 2-3 GB
    Dedicated database server, 6 GB RAM 5 GB
    Dedicated database server, 12 GB RAM 10 GB
    Dedicated database server, 24 GB RAM 18 GB

    Further important MySQL configuration settings

    innodb_thread_concurrency = 2 * [numberofCPUs] + 2
    innodb_flush_log_at_trx_commit = 2
    thread_concurrency = [number of CPUs] * 3
    thread_cache_size = 32
    table_cache = 1024
    query_cache_size = 64M
    query_cache_limit = 2M
    join_buffer_size = 8M
    tmp_table_size = 256M
    key_buffer = 32M
    innodb_autoextend_increment=512
    max_allowed_packet = 16M
    max_heap_table_size = 256M
    read_buffer_size = 2M
    read_rnd_buffer_size = 16M
    bulk_insert_buffer_size = 64M
    myisam_sort_buffer_size = 128M
    myisam_max_sort_file_size = 10G
    myisam_max_extra_sort_file_size = 10G
    myisam_repair_threads = 1

  6. Checkout MySQLTuner

    MySQLTuner is a Perl script that allows you to review a MySQL installation quickly and make adjustments to increase performance and stability. The current configuration variables and status data is retrieved and presented in a brief format along with some basic performance suggestions.

  7. Mount disk with noatime option

    Linux has a special mount option for file systems called noatime. If this option is set for a file system in /etc/fstab, then reading accesses will no longer cause the atime information (last access time – don’t mix this up with the last modified time – if a file is changed, the modification date will still be set) that is associated with a file to be updated (in reverse this means that if noatime is not set, each read access will also result in a write operation). Therefore, using noatime can lead to significant performance gains.

  8. Memory-based filesystem for dynamic data

    By storing dynamic data (var/cache, var/session) on a memory-based filesystem like RAMdisk or tmpfs, the disk I/O is decreased.

  9. Host the shop in same country where your customers are

    The nearest location to your customers means the fastest response.

  10. Install the newest PHP Version

    PHP 5.2.x is very old and slow. PHP 5.3.x is about 30-40% faster than PHP 5.2.x and has a lower memory footprint. PHP 5.4.x is about 20% than PHP 5.3.x for magento.

    Some key parameter for your php.ini

    realpath_cache_size = 32k
    realpath_cache_ttl = 7200
    max_execution_time = 90
    max_input_time = 90
    memory_limit = 256M
    default_socket_timeout = 90
    pdo_mysql.cache_size = 2000
    output_buffering = 4096

  11. Install a byte code cache like APC

    By default, Magento saves it’s cache data in file system. This is fine for small sites with low traffic, but as you get more and more requests, reading and writing to the file system become slower from time to time. With APC you will not have such problems because all valus are stored in memory.

    Example php.ini configuration

    apc.enabled = 1
    apc.optimization  = 0
    apc.shm_segments = 1
    apc.shm_size = 768M
    apc.ttl = 48000
    apc.user_ttl  = 48000
    apc.num_files_hint = 8096
    apc.user_entries_hint = 8096
    apc.mmap_file_mask = /tmp/apc.XXXXXX
    apc.enable_cli = 1
    apc.cache_by_default  = 1
    apc.max_file_size = 10M
    apc.include_once_override = 0
  12. Swap Apache for NGINX

    NGINX is a high performance, high concurrency edge web server with the key features to build modern, efficient, accelerated web infrastructure.
    In one of my previous project (biggest e-commerce site in brasil) with 750.000 unique visitors / day we swapped 6 apache webserver with 2 nginx webserver and we had a load of 1.00 afterwards.

  13. Enable Gzip Compression in NGINX configuration or Apache .htaccess

    Test your Site: http://www.gidnetwork.com/tools/gzip-test.php

  14. Set expiration header for static files

    Configuration for nginx vhost:
    location ~* ^.+\.(css|js|jpg|jpeg|gif|png|ico|gz|svg|svgz|ttf|otf|woff|eot|mp4|ogg|ogv|webm)$ {
    expires max;
    access_log off;
    }
    Apache:
    Make sure the module “mod_expire” is enabled

  15. Apache modules – use just necessary modules
  16. Apache: Eliminating directory structure scans for the .htaccess
  17. Enable KeepAlive

    KeepAlive provides long-lived HTTP sessions which allow multiple requests to be sent over the same TCP connection. In some cases this has been shown to result in an almost 50% speedup in latency times for HTML documents with many images.

    NGINX: http://wiki.nginx.org/HttpCoreModule#keepalive_timeout (Default is 75 seconds, which is good)
    Apache: http://httpd.apache.org/docs/current/misc/perf-tuning.html

  18. Use a Content Delivery Network (CDN) for delivering static files like js, css and images to offload your server

    Afterwards your server has only php request to handle this means all requests for js, css and images are handled by the CDN. You may save a lot of hosting costs.
    My personal experience
    I made very good experience with Amazon Cloufront because it’s very easy to integrate and not expensive and fully integrated in the Amazon Webservice. In my previous posts i wrote about how to use magento with Cloudfront.

  19. Uninstall xdebug or zend debugger on production

    It’s a perfect tool for a development or testing environment but not for production. In some of our projects xdebug had a performance impact of 10-15%, especially with high traffic.

B) Magento configuration

  1. Single Server Environment:
    Use APC as magento cache storage (app/etc/local.xml)

          <global>
            ...
             <cache>
                <backend>apc</backend>
                <prefix>mgt_</prefix>
             </cache>
            ...
            </global>

    Multi Server Environment
    Use memcached as central cache storage

    <global>
    ...
    <cache>
      <backend>memcached</backend><!-- apc / memcached / empty=file -->
      <memcached><!-- memcached cache backend related config -->
        <servers><!-- any number of server nodes can be included -->
          <server>
            <host><![CDATA[127.0.0.1]]></host>
            <port><![CDATA[11211]]></port>
            <persistent><![CDATA[1]]></persistent>
            </server>
        </servers>
        <compression><![CDATA[0]]></compression>
        <cache_dir><![CDATA[]]></cache_dir>
        <hashed_directory_level><![CDATA[]]></hashed_directory_level>
        <hashed_directory_umask><![CDATA[]]></hashed_directory_umask>
        <file_name_prefix><![CDATA[]]></file_name_prefix>
      </memcached>
    </cache>
    ...
    </global>
         
  2. Store sessions in memory
    Single Server Environment:

    Setup Memory-based filesystem for dynamic data (see point 8 above) or use memcached like multi server environment

    Multi Server Environment

    <global>
    ...
    <session_save><![CDATA[memcache]]></session_save>
    <session_save_path><![CDATA[tcp://127.0.0.1:11211?persistent=0&weight=1&timeout=1&retry_interval=15]]></session_save_path>
    <session_cache_limiter><![CDATA[]]></session_cache_limiter>
    ...
    </global>
  3. Uninstall any extensions that you don’t actually use.

    Disable not needed extensions in app/etc/modules/*.xml or delete

  4. Enable all Magento Caches: System -> Cache Management.
    Magento Cache Storage Management

  5. Enable “Use Flat Catalog Category” and “Use Flat Catalog Product” to decrease database queries: System -> Configuration -> Catalog -> Catalog -> Frontend
    Magento Use Flat Catalog Category / Product

  6. Enable “Merge JavaScript Files” and “Merge CSS Files” to reduce the number of http requests
    Magento Merge Js and Css Files

  7. Use a Full Page Cache
    Magento Full Page Cache

    More Information + Demo

    • 90% Faster + 200x More Requests per second
    • Results in higher customer satisfaction and increased sales
    • Amazon: +100 ms, -1% sales
    • Better SEO rating from Google
    • Google webmaster tools report site loading in more than 1.5s as “slow”
    • Study show 70% of revenue come from the fastest visits
    • Dramatically decrease page load time
    • Dramatically decrease your server load
    • Reduce hardware requirements
    • Reduce the number of required web servers
    • Shrink your bounce rates.
  8. Don’t use layered navigation if you don’t really need it, it needs a lot of resources
  9. Don’t use ioncube encrypted extensions

    Ioncube is an encoding system for PHP files. Ioncube-encoded files aren’t readable (by humans) so software companies use it to protect their intellectual property. There are many disadvantages with ioncube encrypted extensions. The biggest disadvantage is that it slow down your shop and it’s impossible to extend. Another big problem is that you can’t step into the code with a debugger like xdebug or zend debugger to see what happens.
    Willem from http://www.byte.nl wrote a nice article about the ioncube and magento performance.

  10. Limit the number of products on a product overview page.
    System -> Configuration -> Catalog -> Catalog -> Frontend

    Magento Limit Products per Page

  11. Disable the Magento log
    System -> Configuration -> Advanced -> Developer -> Log Settings (default is disabled)

    Magento disable log settings

  12. Enable cron and log cleaning: System > Configuration > Advanced > System -> Log Cleaning

    Magento enable log cleaning

    By default, Magento logs every visitor’s requests to the database. As time goes, the number of records may become enormous and slowing down writes as the table grows.

C) Magento Templating/Frontend

  1. Compress images

    We often think that the higher the image quality is, the better our blog/site will look. Unfortunately, you are sacrificing a much more important feat – site’s performance! Higher image quality means higher file size. Higher file size means it will take longer and slower to upload/serve that file to a visitor.

    Recommended services for image compression

  2. Minimize Javascript use
  3. Specify Image dimensions.
  4. Use Block cache and HTML output in your extensions.
  5. Remove all blocks which you actually don’t use/need from layout xml
    Example how to remove the poll from the right column

    <layout version="0.1.0">
      <default>
        <reference name="right">
          <remove name="right.poll"/>
        </reference>
      </default>
    </layout>
  6. Use a Lazy Image Loader to load images on demand
  7. Use a lighweight template

D) Bonus Tips

  1. Install only extensions which you really need, less is more
  2. Avoid ioncube encrypted extensions
  3. Start developing with a developer toolbar from the very first beginning
    Magento Developer Toolbar

    More Information + Demo

    • Requests: involved controller classes, modules, actions and request parameters
    • General Info: website id, website name, store id, store name, storeview id, storeview code, storeview name and configured caching method
    • Handles:overview of layout handles
    • Events/Observer: overview of events with it’s observers
    • DB-Profiler: Number of executed queries, average query length, queries per second, longest query length, longest query and detailed query listing including simple syntax highlighting of SQL
    • Config: enable/disable frontend hints, inline translation and cache clearing
    • PHP-Info output of phpinfo()
    • Profling output of Varien_Profiler with function execution time, function count and memory usage
    • Additional Information: Information: version information, page execution time and overall memory usage
    • Blocks: overview of block nesting

E) Speed testing tools, Other useful stuff

  1. Test the Load Time of a Web Page – http://tools.pingdom.com/fpt/
  2. Google Pagespeed Tool – https://developers.google.com/speed/pagespeed/?hl=en
  3. Firebug integrates with Firefox to put a wealth of development tools at your fingertips while you brows – https://addons.mozilla.org/de/firefox/addon/firebug/
  4. YSlow analyzes web pages and why they’re slow based on Yahoo!’s rules for high performance web sites
    https://addons.mozilla.org/de/firefox/addon/yslow/

Are you interested in a High Performance Solution for Magento?

Checkout the following site for our Managed Cloud Hosting Packages:
http://www.mgt-commerce.com/high-performance-magento-aws-cloud-hosting.html

Magento Cloud Hosting by Amazon AWS

We love hearing from readers, visitors and people from the magento community.
We really appreciate you taking the time to get in touch.

Get in touch with us

11 responses to Magento on Steroids – Best practice for highest performance

  1. Great article. Lots of useful tips. I would also recommend using servers with SSD storage capabilities.

  2. hi
    Can anyone suggest me how to add a reference point in between two toolbars in listing page.please help me

  3. Hi,
    thanks for a good walk-through. However, on a new MySQL installation on Ubuntu Server 12.04, MySQL will not start if you add the parameter myisam_max_extra_sort_file_size. this seems to be an outdated parameter.

  4. great article. thx

    One of the bonus tips is to install MGT_Developer_Toolbar.
    “Start developing with a developer toolbar from the very first beginning”

    Don’t forget to disable (in XML) or uninstall the extension once you’re site is live.
    It’s pretty cool to disable the cache using the extension, but on a live site this is not so cool.

  5. Great info, thanks.

    A lightweight template could indeed help a lot. However, because most modern shops pack a lot of functionality for the visitor most new start ups use heavier templates to be able to compete with the bigger etailers.

    Which templates do you recommended being lightweight and up to modern standards (responsive f.e.)?

  6. WOW just what I was looking for. Came here by searching for magento performance
    optimization

  7. Great article – would be nice to see some stats on how fast you can get a Magento website loading though – page speed wise!

  8. Thank you for great Tips. Good Work.

Trackbacks and Pingbacks:

  1. Optimization tools & resources | indentity - November 7, 2012

    [...] Page Load Time with Site Speed Analytics Report – from Google’s Conversion Room BlogMagento on Steroids – Best practice for highest performance101 ways to speed up your Magento e-commerce [...]

  2. Optimization: links & resources | indentity - November 7, 2012

    [...] Page Load Time with Site Speed Analytics Report – from Google’s Conversion Room BlogMagento on Steroids – Best practice for highest performance101 ways to speed up your Magento e-commerce [...]

  3. MySQL Configuration Changes to boost Magento Performance - September 13, 2013

    [...] As seen on:  http://www.mgt-commerce.com/blog/magento-on-steroids-best-practice-for-highest-performance/ [...]

Leave a Reply

*