Get your copy of the 2022 Gartner Hype CycleTM for Digital Commerce Report.
3 Mins read

How we measure GraphQL response times

Kilka Seller Case Study_Nov 2022
Published on October 4, 2022
Kilka Seller Case Study_Nov 2022

“What response times can we expect from your GraphQL API?” 

This is probably one of the most frequently asked questions we get at Marketplacer in relation to our GraphQL API. It’s a very reasonable question, and we understand why our customers ask it, but it’s not always that easy to answer…

In this article, we share one of the approaches we’ve taken to benchmarking GraphQL API response times at Marketplacer.

But first, some background…

The “problem” with GraphQL…

The entire value proposition of the GraphQL design pattern (for me anyway) is that:

  • It avoids over-fetching
  • It avoids under-fetching

Or, to put it in another more positive way, you get exactly the payload response you want. The ability to write queries and shape your payload to your exact needs is what makes it such an attractive proposition. To illustrate what I mean, let’s take a look at one of the queries we offer at Marketplacer: advertSearch.

An “advert” in Marketplacer is really just a product, but for various reasons I won’t go into here, we call them “adverts” in the Marketplacer GraphQL API.

AdvertSearch allows consumers of our GraphQL API to pull Products hosted in Marketplacer through into whichever upstream system needs them. Two typical use-cases would be:

  • To render a list of products on a search results page
  • To display a single Product Detail Page (or “PDP” in the land of eCom)

These 2 uses-cases have different characteristics:

  • The 1st requires multiple objects, but each with a “small” amount of data
  • The 2nd requires just 1 object, but with a “large” amount of data

In both these cases, you can use the advertSearch query to get this data, but is it safe to assume both queries would have similar response times? Er, no it would not…

This conundrum really brings us back to where we started, and the question: “What response times can we expect from your GraphQL API?” There isn’t really 1 answer…

Query Profiles

In order to attempt to answer the question around response times, we designed a number of “query profiles” that attempt to cover a range of common use cases.

We then run each of these query profiles at a regular cadence using K6, (a very popular load testing framework) and capture the resulting data for subsequent reporting. This not only gives us a fairly decent response(s) to the aforementioned question, but it also:

  • Allows us to track responses times over time
  • Easily introduce new profiles should our customers request them

The current characteristics of our profiles include the following:

  • Query Depth: How many connections are we traversing in the query? This ranges from 1 upwards. The “deeper” the query, the more costly it is, which may have impact on response times
  • Page Size: Requesting a maximum of 10- results per page, as opposed to 1000, has implications for response times
  • Filters: How many filters are being applied with search-type queries?

So for example, here are 2 query profiles for advertSearch, (we have many more but these are just 2 extreme examples):

  • 1-Level Depth / 10 Results Per Page / No Filtering
  • 2-Level Depth / 1000 Results Per Page / 1x Filter

As I’m sure you can imagine, the response time of the 1st query is lower than the 2nd one…

Measurements Taken

As mentioned, we use K6 as the primary vehicle to run tests, with each test being run with the following parameters:

  • Virtual Users (VUs): 1
  • Duration: 10s

These can, of course, be changed, but for benchmarking purposes, we feel this ok. We then gather the following metrics from the result of each test:

  • Median Response Time
  • Average Response Time (averages are not great, but we capture them anyway)
  • p(90): 90% of requests should be faster than the given latency
  • p(95): 95% of requests should be faster than the given latency

K6 has a much wider range of metrics available, but again we just wanted to try to answer a simple question in the simplest way possible…

Final Thoughts

As mentioned in the introduction, we employ some complementary approaches to measuring GraphQL response times; however, this technique gives us a well-defined set of tests that make point-in-time comparisons fairly easy.

Published on: October 4, 2022

Was this article helpful?

Like 1
Dislike 1
graphics graphics graphics

Related posts


Cyber weekend supercharged: Marketplacer powers a sales surge for retailers around the world

Marketplacer has developed a platform that integrates seamlessly with major commercial platforms, including Salesforce Commerce Cloud, Adobe Commerce Cloud, Commerce Tools and Shopify. Using plug-ins makes it possible to get an online marketplace open for business within weeks, in many instances.

December 19, 2022


A technical look at Barbeques Galore’s connected marketplace solution.

Established in 1977, Barbecues Galore - Australia's leading barbeque and outdoor furniture retailer - has long invested in its digital capability, early on expanding from their traditional brick-and-mortar presence into eCommerce. But in recent years, this single step has not been enough to keep pace with consumer behavior and retailers have been shifting their approaches accordingly.

December 16, 2022


Marketplacer unlocks the retail media opportunity for marketplaces globally with CitrusAd partnership


[Sydney and Brisbane], 1 December 2022 – Marketplacer, a global technology platform that enables brands, retailers, suppliers, communities and innovators to easily build and grow successful online marketplaces at scale, today announced it has signed a partnership agreement with world-leading retail media tech company, CitrusAd, to deliver advanced advertising opportunities.

December 8, 2022