BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News FlexSearch.js: A Fast, Zero-Dependency Full-Text Search Library

FlexSearch.js: A Fast, Zero-Dependency Full-Text Search Library

This item in japanese

Bookmarks

Nextapps recently released FlexSearch, a full-text, zero-dependency JavaScript search library for web browsers and Node.js. FlexSearch strives to be the fastest search library available to JavaScript developers through its scoring algorithm. FlexSearch also provides developers with memory optimization options.

FlexSearch benchmarked its search against eight other search libraries, including Wade, Fuse, and Lunr. Under the fast preset for this benchmark, FlexSearch performed, in a given amount of time, 300 times more search operations than Wade, the next fastest search library. The benchmark is based on the book Gulliver's Travels. Benchmark results differ depending on the conditions of the run (e.g. machine specifications, operating system, and available memory).

FlexSearch attributes its speed differential to Contextual Search, a scoring algorithm created by Thomas Wilkerling, FlexSearch's main contributor. The basic idea of Contextual Search is to compute relevance within a limited context whose depth can be customized, instead of computing the relevance on an entire result set. FlexSearch hence manages a contextual index, which consists of an in-memory pre-scored dictionary as its base. As a result, the speed gained with Contextual Search comes at the expense of the memory footprint, which grows together with the configured depth for the context.

FlexSearch can be customized extensively. The profile option can be set to optimize for search speed (fast), or memory consumption (memory). Other options allow for configuration of contextual depth, relevance threshold, word tokenization, stemming, encoding, caching, or to enable web workers. Through a careful configuration of options, developers can optimize searches for memory consumption. FlexSearch benchmarked memory-optimized searches against seven alternative libraries, showing that FlexSearch's memory-optimized searches presented the lowest memory consumption among these alternatives under this test.

FlexSearch can be used in web browsers and Node.js. Prior to performing a search on searchable content, developers must create an index, and populate that index with the searchable content (strings, or documents). Developers can then use the populated index to search for content. The following example shows a basic speed-optimized search:

var index = new FlexSearch("speed");
index.add(10025, "John Doe");
index.search("John");

For searching a small amount of simple textual data (like product data for a small online shop), developers can use string matching without resorting to a full-text search library.

Typical basic full-text search features include:

  • tokenization: strings are broken into words differently across languages
  • stemming: caring for elders matches results for care for elders
  • stopword handling: avoid irrelevant results caused by common words such as a and the
  • basic fuzzy matching: service workers matches results for Service Worker

Advanced full-text search features include:

  • autocomplete: typing progre… invokes suggestions including Progressive Web App
  • suggestions: typing Porgessive Web Apps suggests Progressive Web Apps
  • search expressions: enables search to use AND, OR, NOT operators
  • phrase matching: enables to control how much distance can be allowed between words in a search phrase. For instance, one may configure a search for black friday to match the exact phrase black friday or to match also descriptions that include the words black and friday
  • synonym search: typing shoes or footwear should return similar results

Most of the benchmarked full-text search libraries provide the aforementioned search basic features. FlexSearch additionally provides advanced features like suggestions, partial matching, phonetic matching, pagination, and search expressions. FlexSearch does not however provide synonym search.

FlexSearch is open source software available under the Apache 2.0 license. Contributions are welcome via the FlexSearch GitHub project.

Rate this Article

Adoption
Style

BT