Behind the Scenes of Jetpack Search: An Exclusive Interview

Jetpack Search has proven to be quite popular among WordPress site owners — and for good reason. This relatively new tool has the magical ability to understand the true intent of a visitor and serve up exactly what they’re looking for, all in a split second. Sometimes the perfect result is displayed even before the searcher has finished typing.

The outcome? People stay on sites longer. They read more, view more pages, and buy more products. 

If you haven’t read Kylie M. Interiors’ story, you should make the time. This incredible business from a small town in Canada receives more than 50,000 page views each day. They needed a reliable search plugin and their dreams came true with Jetpack Search. From the article:  

Kylie’s husband of 15 years, Tim, handles business operations and runs their WordPress site.  “At the end of it all, Jetpack Search was the best solution I found,” Tim said. “We’re euphoric over how well it works.” 

“Tim didn’t tell me that he changed the search function,” Kylie added. “One day I made a search and was like, ‘I got what I wanted!’”

Kylie described the new search functionality as a ”game changer” for her business. “Now you can find any of the 300 articles you’re looking for in a hot second,” she said. “I’m advertising it in my client consultations and telling people to use it because it works.”

Given the success of the plugin, we wanted to shine a light on the heroes behind the scenes. How exactly does it work? What’s the story behind it? 

We interviewed Jason Moon, who wrote most of the code for Jetpack Search, as well as other team members Chris Rosser and Robert Felty. We also got feedback from Greg Brown, who helped architect Jetpack Search and bring the idea to fruition.

An interview with the Jetpack Search team: behind the development

What was the thinking behind Jetpack Search? Why was it important to create a tool like this for WordPress?

Jason Moon: 

I think many people seek out Jetpack Search for two big reasons: performance and functionality. 

Let’s start with performance. As a WordPress site accrues searchable records over time, the built-in search functionality may deteriorate. This is primarily due to WordPress search’s reliance on MySQL, which isn’t a particularly fast solution for the free text search that we’ve been accustomed to on the modern web.

We have a saying among product designers and engineers: “Performance is user experience.” When you present your users with a lackluster search experience, it has a ripple effect across your business, negatively impacting your brand and the bottom line.

By offloading this performance issue onto Jetpack Search, we take care of it on your behalf. Behind the scenes, we use Elasticsearch to index and serve searchable records to your visitors near-instantaneously, no matter how many a site might have. 

Another big reason a site owner might seek us out is for the added search functionality lacking in WordPress core. We released a new iteration of Jetpack Search last year, complete with a modern yet lightweight JavaScript interface that meshes well with any WordPress theme. This interface features a search-as-you-type experience that dynamically updates results as you tweak various parts of your query, including your search term and filters.

Who’s responsible for the idea and approval to make this happen?

Jason: 

The leading advocate for search in Automattic has to be Greg Ichneumon Brown, who has been a critical figure for all search-related initiatives in the company. Although he’s not currently with us on the Jetpack Search team, Greg remains one of our key technical advisors for product decisions.

Greg also spearheaded the formation of what eventually became the Jetpack Search team, so we owe him a lot of credit.

Does developing this tool as part of Automattic give you certain advantages (resources, structure, etc.) compared to other organizations? 

Greg Ichneumon Brown: 

Because WordPress.com already serves a vast amount of traffic and runs its own data centers worldwide, Jetpack Search is able to build on all that existing infrastructure. While many similar solutions require running on fairly expensive cloud platforms (AWS or Google Cloud), Jetpack Search is able to significantly reduce and simplify our pricing model since all of the systems involved are our own.

Jetpack Search also relies heavily on the stats from your site about what is popular to better match users to the content they’re most likely to engage with. We’ve been running Jetpack Stats for over ten years and that infrastructure can be expensive and tough to scale for larger sites.

The other benefit here is privacy. Serving both search content and stats through the same systems means that you’re only sharing your data with a single company. Getting the same systems from other companies often means sharing your data with multiple entities and almost always with Google and Amazon. We’re proud to offer an alternative and we take a very different stance towards privacy.

How many people contributed to development? How long did it take? 

Jason: 

While search has been a core competency for Automattic for well over a decade, our latest development of Jetpack Search began in earnest in late 2019. Initially, Jetpack Search was just one of the projects in the Data Products team at Automattic. Over time, we spun off into a standalone team.

I don’t think we’ve ever had more than five people working on the project simultaneously. The project team initially started with three people and, while colleagues have come and gone in the past two years, we’ve managed to keep our team size the same! 

Any cool stories from the development process?

Chris Rosser:

We had our last pre-pandemic meetup in Boulder in December 2019, attended by the search team, a trio of designers and Jason’s amazing dog Tera, who became our project mascot. It was a productive and really enjoyable few days, and our conversations there helped us figure out the last few pieces of the Instant Search puzzle. Although we’re a fully remote company, meetups have always been part of the Automattic formula. That week is a good reminder of how useful in-person collaboration can be and that we all really miss seeing each other in real life.

Any future goals for the feature? How is it maintained today?

Jason: 

We currently have two projects in the pipeline for Jetpack Search: 

1. Improved integration with WooCommerce. Our team is a big fan of WooCommerce and would love to make product filtering a top feature.
2. Enhanced customization using Gutenberg blocks. We strongly believe that Gutenberg is the future of WordPress customization and, as such, plan to enable configuring Jetpack Search using blocks as soon as possible.

As for maintenance, we have close contact with our colleagues in support, who keep us abreast of any user issues in the field. In addition to customer feedback, we also have an internal list of bugs and features to shape our roadmap.

Jason, as the person behind most of the code, what past experiences prepared you for this project? 

Jason: 

I’ve had the fortune of cutting my teeth with startups early in my career, leading to a strong generalist mentality when approaching new problems. However, I’ve specialized heavily in JavaScript development for the past four years or so, focusing on building out customer-ready interfaces. As such, I was responsible for architecting and building out most of the code that makes up the bulk of our newly-released Jetpack Search product. 

Much of the Elasticsearch infrastructure necessary for Jetpack Search was already in place when I joined the project. We’ve had excellent support from search technology and Elasticsearch experts at Automattic, which enabled our team to focus primarily on the user experience.

An interview with the Jetpack Search team — behind the Tool 

What would you say makes Jetpack Search different from other WordPress search tools?

Jason:

I’d say our most significant differentiator is the fact that we’re a hosted solution. We offload any search-related work from your WordPress server onto our cluster of servers distributed worldwide. Our infrastructure, which routinely handles 20 billion requests each month, ensures that your visitors will always enjoy a snappy search experience. 

How easy is it for beginners to set up without knowing code?

Jason:

Approachability and ease of use are our top product goals for Jetpack Search. Getting started is as easy as installing the Jetpack plugin and purchasing a Jetpack Search subscription. From there, we take care of everything necessary to move forward, ranging from indexing your site to setting up your initial search filters.

However, we realize that a one-size-fits-all approach is not a good solution for most of our customers. To address this, we also have extensive customizability built into Jetpack Search. Site owners can tailor various parts of the search experience for their audience. An online storefront, for example, might find our grid-based “product” result layout to be a significantly better fit than our row-based “minimal” layout. 

What types of algorithms does Jetpack Search use to determine results?

Robert Felty:

Jetpack Search looks at all the content of a particular post, including the title, excerpt, body, comments, tags, and more. We then use industry-standard ranking algorithms to measure the relevancy of a post for a given search term.

For example, if you search for “the cat,” we look for all posts with “the” and “cat.” However, since “the” is such a common word in English, it’s weighted significantly less than “cat.” We also look at how frequently the search term occurs within each post. If post A mentions “cat” several times, while post B only mentions it once, we give a higher weight to post A. We also use data from Jetpack Stats to consider the number of times a post has been recently viewed to try to return the most relevant search results to the user.

How do filters work?

Jason: 

Currently Jetpack Search allows you to filter results by date, taxonomy (i.e. category/tag), and post type. This includes custom post types and custom taxonomies. It’s all configurable to your tastes as well, so if you use tags on your site, but not categories, you can display the option to filter by tag, but not by category, or vice-versa, or both! And let’s not forget that this filtering is also lightning fast, due to smart caching of filters.

How does real-time indexing work? What are the benefits of this?

Greg: 

This is a wonderful question — I could write an entire post just on this. Actually, I have. I might be biased, but if you’re interested in learning more about real-time Elasticsearch indexing on WordPress.com — this is a great place to start. 

Can you tell us more about automated spelling correction? How does Jetpack Search know what to correct? 

Jason:

When you search for a term that doesn’t return any results, we search for similar words automatically, and return those results instead. For example, if you have a food blog, and someone searches for “vegetible” (misspelled), Jetpack Search will instead return results for “vegetable” (correctly spelled). 

What types of sites would benefit from Jetpack Search the most?

Jason:

If your site search is beginning to slow down, I think it’s a perfect time to give Jetpack Search a go. Depending on your hosting specifications, you might run into performance issues with just a handful of pages! Here’s my quick rule of thumb to check for this bottleneck: If your site search is taking more than half a second to load, it’s time to look for another solution.

Alternatively, suppose you’re dissatisfied with the static user experience provided by the built-in search. In that case, Jetpack Search could be a great fit for you. Our interface delivers a universal design compatible with any WordPress theme using cutting-edge web paradigms like reactive rendering.

We like to believe that we’ve priced Jetpack Search competitively and that our performance-first approach resonates with customers looking for a modern, instant search experience.

These are the droids you’re looking for

It’s easy to see the passion behind the development of Jetpack Search. A lot of love was poured into its development, and a lot of love is still at work today. Built on Automattic’s own infrastructure, it’s nimble, efficient, and secure. Plus, it provides accurate (and useful!) results — super fast. 

Did we mention it’s also incredibly affordable? Give your site an advantage — get Jetpack Search, today. 

---