Real Troubleshooting Workflows, Worked Query by Query
Graylog
Chapter 8 · Real Troubleshooting Workflows, Worked Query by Query
No new syntax in this chapter — just three realistic support scenarios, each worked from the first broad query through to a specific answer, using nothing but the fields and techniques Chapters 2 through 7 already covered.
Scenario 1: "A Customer's Search Results Look Wrong"
A ticket reports that account bullhorn-27515 is seeing unexpected search results on the
legacy Daxtra Search product. Following Chapter 7's own incremental-building method:
Step 1 alone confirms the account is genuinely active on the right service — a real, useful sanity check before assuming anything is broken. Step 2 filters straight to anything worth investigating. Step 3 follows Chapter 4's own request-correlation pattern to see the full sequence of what happened around that one specific error.
Scenario 2: "RMS Integration Jobs Are Failing Overnight"
A report comes in that RMS integration jobs failed overnight, with no specific account named yet.
Starting at the cluster level (Chapter 2) rather than guessing at a service or account first is the
right call here — the ticket doesn't name either, but it does name infrastructure. Once step 1 surfaces
a specific failing job, Chapter 4's own correlationId — not requestId — is the
right field to reach for, since this is a cross-service Flux-style job, not a single HTTP request.
Scenario 3: "Scout Errors on One Region"
Monitoring flags an increase in scout errors, specifically on the eu-west-2 cluster.
level:<3 here matches only levels 0, 1, and 2 — in
real practice at logger.daxtra.io, that's critical-only, since 0 and 1 aren't populated. If the report
of "errors" genuinely includes level-3 Error messages, not just level-2 Critical ones, this query is
silently too narrow. The corrected version, matching Chapter 3's own fix, is
level:(2 OR 3) in place of level:<3 — always worth double-checking which
one a report actually means before trusting the result count.
| Scenario | Starting field | Refinement field |
|---|---|---|
| Search results look wrong | service + account | level, then requestId |
| RMS jobs failing overnight | daxtraResource (cluster) | level, then correlationId |
| Scout errors, one region | service + daxtraResource | level range — check < vs. OR-grouping |
Hands-On Exercises
In Scenario 1, explain why Step 1 (service + account, no level filter yet) is worth running before Step 2, rather than jumping straight to the level-filtered query.
📄 View solutionIn Scenario 2, explain why correlationId is the right field to reach
for in Step 2, rather than requestId.
Rewrite Scenario 3's own query so that it genuinely includes both critical (2) and error (3) level messages, using this chapter's own corrected form.
📄 View solutionChapter 8 Quick Reference
- Scenario 1 — service + account confirms real activity first, then level narrows to problems, then requestId follows one specific error
- Scenario 2 — daxtraResource (cluster) is the right starting point with no named service/account yet; correlationId (not requestId) traces a cross-service job
- Scenario 3 — a real, worked reminder that
level:<3excludes level 3 itself; confirm what "errors" means before trusting the query - Every scenario reused only fields and techniques already covered in Chapters 2-7 — no new syntax needed
- Next chapter: Avoiding False Signals & Performance Pitfalls