How do I make search-as-you-type fast with edge n-grams in Elasticsearch?

Short answer: drop wildcard/regexp; move the work from query time to index time — build the prefixes when you index with edge_ngram so autocomplete becomes a plain term lookup.

Short answer

Here’s the real issue: wildcard/regexp are scans that run at query time; they can’t use the inverted index efficiently, so on every keystroke they try to filter 10M documents and burn CPU. The fix isn’t to speed up the query, it’s to do the work ahead of time. For an introduction to Elasticsearch’s indexing and analysis model, see getting started with Elasticsearch and Kibana.

Why

1. The gap between a scan and a lookup is a gap in scale. edge_ngram emits every prefix of a word (“a”, “ay”, “ayk”, “ayka”…) as separate tokens at index time; when the user types “ayk” there’s no scan, the token is looked up directly. All the speed of as-you-type comes from here.

2. The most common mistake is not separating the analyzers. If you ngram the query too, every prefix the user types gets re-split, producing irrelevant matches and inflated scores.

3. Ngrams inflate the index exponentially. Keeping the gram range wide pays back the speed you gained in disk and RAM.

What to do

1. Apply edge_ngram only on the index side. Let analyzer build ngrams at index time and search_analyzer stay standard on the search side. The index builds the prefixes, the search just matches them.

2. For Turkish, add lowercase + asciifolding. Put these before the ngram in your custom analyzer so “Şişe” matches “sise” and “Ürün” matches “urun”. On a Turkish search bar this isn’t negotiable.

3. Keep min_gram/max_gram tight (e.g. 2-15) and limit which fields are searched.

4. Add debounce on the client (e.g. 150 ms); don’t fire a request on every keystroke.

5. Consider the shortcut. ES’s search_as_you_type field type sets this up (edge ngram + shingle) for you.

Bottom line: I’d build autocomplete at index time, either with a custom edge_ngram analyzer or the ready-made search_as_you_type field; I’d never use wildcard for autocomplete. Ngram + lowercase + asciifolding on the index analyzer, standard matching on the search analyzer, keep the gram range tight, and add debounce on the client. Done right, search across 10M products drops to milliseconds per keystroke.