Skip to Main Content

4 Website E-Commerce Testing Strategies for the Holiday crush

Posted on 1.13.2015

:: By Archie Roboostoff, MicroFocus ::

The holiday season is over, which is both good news and bad news for leading online retailers. While the period between Thanksgiving and Christmas generates a healthy portion of annual online sales, it also can be a chaotic stress test for IT decision makers charged to ensure a smooth, non-disruptive e-commerce experience for consumers accessing sites across a broad range of desktop and mobile devices. 

A key takeaway from the 2014 online shopping season is that instead of marketing efforts designed to drive all major sales to Black Friday and Cyber Monday, retailers extended promotions and deals across several days. As a result of this shift, leading US retail websites – including Amazon, Apple, Best Buy, Home Depot and Urban Outfitters – not only saw their website performance suffer on Black Friday and Cyber Monday, but also struggled with peak web traffic throughout the holiday shopping season. 

For example, according to new retailer website performance data compiled by Micro Focus’ website performance tool, Borland Silk WebMeter,’s US website slowed from approximately 1 second load times the week before Thanksgiving to peaks exceeding 10 seconds on 11/24; 12 seconds on 11/25; 21 seconds on 11/26; and 10 seconds on 11/30. None of these excessive load times occurred on Black Friday or Cyber Monday. 

The e-commerce challenges many brands faced during the holidays serves as a reminder of the need for sound peak load testing strategies – not only to handle traffic on traditional online shopping promotional days but throughout these newer, extended periods of heavy traffic. Nearly half (47%) of consumers expect a web page to load in 2 seconds or less, and after 3 seconds, 40% of these shoppers will abandon the site, according to KISSmetrics. And once visitors leave, it is extremely difficult to get them back: 88% of online consumers are less likely to return to a site after a bad experience. 

As a result, failure to develop and implement peak load testing strategies can lead to websites that crash, slow down to an unusable pace, or failed to prepare for increased consumer usage of mobile sites to make purchases. 

The inevitable question when brand websites and e-commerce capabilities fail during at the most critical moment is…why? Most firms will insist they measure, test, and measure again. And many do; but for a number of reasons the website failures during peak load periods are as predictable as the neighbor down the street who puts up their Christmas lights before the calendar turns to December. 

Challenges to peak load testing

The majority of sophisticated brands measure and test website performance in advance of the holidays and other high traffic periods in order to ensure websites (desktop and mobile) can handle the load. However, some underestimate the load, expecting, for example, a 40% jump and instead getting an 80% increase. Beyond underestimating the website traffic spike, the use of traditional performance testing software has required a significant investment of money, time and resources – serving as a barrier to adoption and leading some to ignore the peak load challenge altogether. 

Chronic departmental silos add another layer of complexity to the problem. Traditionally, website performance was considered an IT problem, but as marketers become increasingly tech–savvy, a poor performing website is a big problem for the marketing department as well. A failed website can have detrimental effects on seasonal promotions and sales. Often the website peaks and troughs are driven by marketing activity, but marketers need to ensure that their website can handle an increase in traffic and is ready for peak performance. 

Effective performance testing of websites and mobile applications for the peak load of intense shopping periods requires several best practices, including: approaching the problem strategically; testing continuously and moving testing forward with planning; realistic testing that accounts for usability, behavior patterns, device types, network speeds and location; virtualized testing environments; and elastic execution.  

More specifically, retailers should consider the following four website and mobile testing strategies to ensure they maximize the revenue and business opportunity the holiday season and other promotional periods offer – and adequately prepare for the next planned or unplanned spike. 

1. Performance test regionally

Given the global nature of today’s market, it is critical to capture the performance experience of customers across all potential access regions.  There are so many variables in play based on the physical location of the servers, and if you are only testing users in a confined geography, you are opening the door to unforeseen issues. An organization needs to know what the performance experience is for all users coming to their sites.  By segmenting performance experiences across various regions, an organization will be able to come up with options to overcome any degraded performance that might result from implementing more servers, streamlining “foreign” transactions and/or adding local content delivery networks.  

2. Test adequately for mobile

In January 2014, comScore noted that, for the first time, mobile web browser usage overtook desktop web browser usage in the United States. Testing for the user experience is more difficult than ever before, as there are more devices, more platforms and more browsers. 

Walmart recently became the latest major retail brand to transform its online shopping application with a mobile focus designed to increase conversion rates on the application and improve the shopping experience for mobile customers. 

The better the mobile experience, the more consumers will be inclined to do their shopping on smartphones and tablets. Retailers must increase their performance testing across mobile properties to ensure that all of the hard work and money spent to increase mobile usage is not undone by sites that don’t load or apps that crash during peak periods.  

Aside from testing the mobile end user experience, it’s also important to simulate various mobile network speeds.  In some circumstances, servers have to work harder to deal with items like mobile latency so a mobile network speed simulation should also be conducted in conjunction with overall performance testing.  This can be done by allocating a certain percentage of users in the overall performance test to, for example, originate from a 3G network using an iPhone 5.  This will isolate where the application isn’t being responsive with relatively slower mobile networks.  

3. Catalog and segment transactions by developing an accurate user model

Different transactions cause the infrastructure to react in different ways and it’s important to segment and catalog key transactions to test appropriately.  Start by developing an accurate user model(s) that reflects how your customers use the site on both an average day and a peak volume day. Essentially, this will become your standard performance test(s). Consider browser types, device types as well as load amounts across the site.  The key is to prioritize performance based on user types, commercial needs and devices.  This allows the company to simulate various transactions to isolate slow processing times and also places an emphasis on business priorities.  This also isolates any 3rd party services that may be slowing the overall site down like analytics and tracking services.  

4. Scale with the Cloud

By implementing a performance testing solution via the Cloud, organizations can more effectively and affordably manage heavy loads on the company’s website.  By using the Cloud, organizations are able to simulate as many users as they want without having to setup or invest in hardware to generate the load.  

Cloud-based performance testing provides a multitude of benefits to support the retailer year-round, and particularly during peak-load periods: 

Assured performance – Cloud-based infrastructures are extremely well suited to generating the peak demands required for enterprise performance testing. Peak load testing in the Cloud takes advantage of the ability to run tests virtually on-demand.  Businesses can simply schedule time for a test and resources are automatically provisioned. 

Worldwide readiness – Using cloud technologies can also enable the performance management team to not only evaluate an application’s global readiness but conduct tests across the globe by replicating virtual users in a variety of different locations to ensure the website can handle users far and wide. 

Cost control – The elasticity of the Cloud provides the ability to scale computing resources up or down as needed to ensure website performance is affordable. Using utility-style pricing, businesses only pay for what they use. In comparison to a traditional on-premise model, a company would have to acquire computing power to support very large user tests for the lifetime of the application.

In a 2014 survey of online sales, Deloitte forecast a tangible change in the narrative of holiday sales. The firm sees traditional retailers starting to catch up to online-only competitors due to significant investments in websites, mobile apps and tablet capabilities. As these brick-and-mortars put more eggs in the online sales basket, consumer expectations will continue to rise. Traditional and emerging retailers that do not test properly for peak load traffic risk losing revenues and customers. 

Archie Roboostoff is Director, Borland Portfolio at Micro Focus, a leading global provider of application modernization software 

Editor's Note: Article updated on Jan. 22.

Today's Top Picks for Our Readers:
Recommended by Recommended by NetLine

Leave Your Comment

Login to Comment

Become a Member

Not already a part of our community?
Sign up to participate in the discussion. It's free and quick.

Sign Up


Leave a comment
    Load more comments
    New code

    The Ultimate Guide to Personalization