Using Magento as the coal-miner's canary

Overview

Regular review of web-application logs is not only a requirement for various compliance regimes (such as the PCI-DSS or various IRAP-based programs), it can actually give you good insight into vulnerabilities which arise outside of the web-application itself. In this post we describe how clever analysis of blocked-request logs (in this case to Magento) actually provided us with information about compromised computers elsewhere within our client’s organisation.

Magento is an e-commerce platform which drives the public web sites of many big Australian retailers. Magento has an administrative area (which typically lives under the /index.php/admin/ URL space). While this administrative area requires authentication to be accessed, it is prudent for Internet-facing sites to further restrict access to the administrative area by methods that include blocking connections by source IP address. This helps mitigate risks associated with compromise of administrator credentials, where, for example, such compromise may be due to:

  • A compromised desktop used by the administrator, or,
  • Re-use of credentials across different web platforms, one of which has been independently compromised.

As part of a recent secure-Magento deployment, DotSec implemented restrictions to the admin area, and also implemented a managed logging and monitoring service.  Almost immediately, the managed logging and monitoring service began to reveal that while the source IP restrictions were working as designed, the admin requests that were being blocked had a very noteworthy and worrying format. Specifically, the blocked requests looked like this:

/index.php/admin/catalog_product/edit/store/0/id/7986/key/5397fc4102cc23cdf3dd1f4e512896671297a1222e13d95acd16a11bbb883234/

The string “5397…3234” in the above URL is known as a form_key.  Form_keys are unique (random) strings which should be impossible to guess;  they are used on the Magento platform for Cross-Site Request Forgery protection, preventing administrative users from being tricked into executing sensitive commands just by visiting (directly or indirectly through javascript includes) malicious websites.

What’s going on?

Clearly an attacker was trying to compromise the Magento server by re-using a what appeared to be a valid form_key! 

But form_keys are effectively impossible to guess so if the form_key was valid, then the attacker in Russia must have a way of somehow to stealing valid form_keys!  How on earth could that happen?

The key to the answer lies in effective logging, monitoring, alerting and reporting.

There are a number of ways that a valid form_key could have leaked into the hands of an attacker;  time to check the logs:

  1. The Magento administrator may have fallen victim to a cross-site scripting (XSS) vulnerability on the Magento site, and the form_key may have leaked out as a result.  In order to understand whether or not XSS was the culprit, it would be helpful if the site-owner had performed a recent penetration test to determine whether or not vulnerabilities (including XSS) were present in their Magento site.  Unfortunately, the client had conducted no such test and so we needed a different approach.
  2. If an XSS attack had taken place, the XSS event which exfiltrated the form_key would have been visible in victim’s browser and/or HTTP proxy logs.  Unfortunately, it was difficult to get permission to access the Administrator’s local-browser history, and there were no proxy logs available.

Ah well, perhaps the form_key leak wasn’t due to XSS… perhaps the form_key wasn’t even valid!  We needed to think what other options we could consider.

Given that form_keys are completely random, the likelihood that an attacker would be able to guess and attempt to use a particular valid form_key is extremely low.  However, the monitoring service showed that in fact, the same form_key was used in a request that was blocked, but which came from a non-white listed IP address in Russia.

Check the logs!

That’s it!  Dotsec manages the Web Application Firewall (WAF) that provided Internet-facing protection for the Magento site. The WAF was configured to log events to a customer-owned Splunk logging infrastructure, and it was logging both allowed and blocked requests, together with all request parameters.  We got access to the customer’s Splunk server and set about cooking up some searches to see if the WAF could enlighten us as to what was going on.  Here’s what we found:

  1. The blocked request wasn’t a once-off event.  There were plenty of similar blocked requests that were being made to protected URLs from non-white listed IP addresses.
  2. A day or so before each blocked request, the Magento site accepted a requested from a white-listed (allowed) IP address.
  3. The form_key in the blocked request was always the same as the form-key used in the previously allowed request.

And so we were on the trail!  The allowed requests were coming from an internal corporate desktop associated with the allowed (white-listed) IP address. And every time the Magento admin logged on from that desktop, someone in Russia attempted to access the protected Magento resource using the form_key that had appeared in the Magento admin’s request. Almost certainly then, malware is running on the Magento admin’s desktop and is leaking sensitive information out to the Internet.

Lessons learned

So, what have we learned?  You can and should take an in-depth approach when securing your Magento site: 2FA and source IP address-restriction are no-brainers.

That’s not enough however and a defence-in-depth approach is required; one that includes other no-brainers like patching together with more orthogonal controls like extensive and automated logging, monitoring, reporting and alerting; anomaly identification and IDS, regular testing and assessment; network controls and a WAF that monitors and controls all traffic associated with the site; and a solid risk-management plan for supporting services such as Admin desktops.  Taking this kind of holistic approach will allow you to be aware of even the most unexpected attacks.  And even when one compromise (like an Admin desktop) is successful, the security of your other assets (like your Magento server) is more likely to remain unaffected.

Have a look at some of our case studies and give us a call if you’d like to see how DotSec can help you do more business, more securely.