Building Effective Lucene Queries

Graylog

Chapter 7 · Building Effective Lucene Queries

Chapters 2 through 6 built up a real field vocabulary. This chapter turns that vocabulary into genuinely effective queries — how to combine fields well, a real precedence trap worth knowing about straight from Lucene's own documentation, and how to exclude results deliberately rather than just include them.

Combining Fields: The Building-Block Approach

The real, most common query shape in day-to-day use is a chain of AND-joined field matches, each one narrowing the result set further. Two real, worked examples:

-- Critical/error logs for one account
account:bullhorn-27515 AND level:(2 OR 3)

-- Critical/error logs for one service
service:scout AND level:(2 OR 3)

Both queries follow the identical real shape: one field pinning down where to look (account or service), combined with level:(2 OR 3) pinning down how bad — the same real severity grouping covered in Chapter 3.

Operator Precedence: Always Parenthesize Mixed AND/OR

A real, easy-to-miss fact straight from Apache Lucene's own documentation: OR is the default conjunction. If two terms sit next to each other with no operator between them at all, Lucene treats that as OR, not AND.

Mixing AND and OR without parentheses is genuinely ambiguous

Lucene's own documentation doesn't guarantee a specific precedence when AND and OR are mixed in one query without grouping. Its own recommended real example: (jakarta OR apache) AND website — explicit parentheses "eliminate any confusion" about which combination was actually intended. Applied to this course's own real field vocabulary: don't write service:scout OR service:search AND level:3 and assume you know what it means — write (service:scout OR service:search) AND level:3 instead, so the grouping is unambiguous regardless of how the query engine happens to resolve unparenthesized precedence.

Excluding Results: NOT and the Minus Operator

Two real ways to exclude a term. The NOT operator removes a specific term, but genuinely requires at least one other term alongside it — NOT can't stand alone as a whole query. The minus operator (-) does the same job and can be used more freely:

service:scout AND NOT level:7
service:scout -level:7

Both real forms above do the same thing — scout's own log activity, excluding debug-level (7) noise.

Free Text + Field Combos

Quoted free text (Chapter 1) and field matches combine the same way any two field conditions do — a real, common pattern for chasing one specific known error message without it getting lost across every service that happens to log something similar:

"connection timed out" AND service:hwsdb

Building a Query Incrementally

In real practice, the most reliable way to build a query is rarely to write the whole thing at once. Start with one broad scoping field (service or account), run it, confirm there's real log volume to work with, then add level, then narrow further with a correlation ID or free-text phrase once you've spotted something specific worth chasing. Each step is independently checkable, which makes it far easier to tell whether an empty result set means "nothing happened" or "one part of this query is wrong."

Goal	Pattern
Scope + severity	field:value AND level:(2 OR 3)
Mixed AND/OR	(field:A OR field:B) AND field:C — always parenthesized
Exclude a value	field:value AND NOT field:other / field:value -field:other
Known error, scoped	"exact phrase" AND service:name

Hands-On Exercises

Exercise 1

A query needs to match either service scout or service search, but only at critical or error level (2 or 3). Using this chapter's own real parenthesization rule, write it correctly.

📄 View solution

Exercise 2

Write a query, using this chapter's own minus-operator syntax, that finds all ws-lite activity except debug-level (7) messages.

📄 View solution

Exercise 3

Explain, in your own words, why building a query incrementally — one field at a time — makes it easier to diagnose an unexpectedly empty result set than writing the whole query at once.

📄 View solution

Chapter 7 Quick Reference

Real, common pattern: field:value AND level:(2 OR 3) — a scoping field plus a severity group
OR is Lucene's default conjunction when no operator is given — mixing AND and OR without parentheses is genuinely ambiguous; always group explicitly
NOT requires at least two terms; the minus operator (-) does the same job and can be used more freely
Free text and field matches combine the same way any two conditions do: "phrase" AND field:value
Build incrementally — one field at a time — to make an empty result set easy to diagnose
Next chapter: Real Troubleshooting Workflows, Worked Query by Query