GrooveSeek

Semantic search over a Markdown knowledge base, served over MCP.

View the Project on GitHub alphabet-h/grooveseek

Architecture

Source-level structure and data flow of groove, for contributors extending or modifying the codebase.

日本語版: ARCHITECTURE.ja.md

Source layout

File Responsibility
grooveseek/src/lib.rs (v0.7.1+) Library crate root. Re-exports the modules below as grooveseek::* so benches under benches/ and integration tests under tests/ can drive internal APIs without subprocess. The library surface is intentionally unstable and not intended for external consumers.
grooveseek/src/main.rs Binary entry point. The clap CLI dispatches every subcommand groove has; each is documented in usage.md, and the service verbs under its service section. Consumes the lib via use grooveseek::*;. Loads groove.toml and merges with CLI args. JSON / text output formatting.
grooveseek/src/config.rs 4-tier groove.toml discovery (--config flag → CWD → .git ancestor (CWD + up to 19 ancestors) → binary-side legacy). Config::discover() returns a ConfigSource enum that main.rs logs at startup. Resolves CLI > config > default precedence. Injects FASTEMBED_CACHE_DIR env when the config sets it and the env is unset.
grooveseek/src/server.rs rmcp::ServerHandler impl. Dispatches six MCP tools. search routes to db.search_hybrid and wraps the result in a SearchResponse with low_confidence / match_spans / filter_applied (BREAKING in v0.3.0; see CHANGELOG). State lives in KbCore; each tool is a thin async wrapper that runs its body there via spawn_blocking, so a long call occupies a blocking thread instead of an async worker.
grooveseek/src/server/search.rs (v1.0.0+) The search half: the search tool body, the pipeline it runs, and the limits a request is held to before either of them sees it. Split out of server.rs the way db.rs was split before it — bodies byte-identical and in the order they were already in, mod tests left in the parent. The only thing that changed was visibility: three private items became pub(super) because the parent still calls or names them. What stayed behind is the tool surface itself, the #[tool_router] / #[tool_handler] impls and the parameter and response types.
grooveseek/src/server/documents.rs (v1.0.0+) Reading a document out of the knowledge base, and deciding whether it may be read at all: the get_document and get_best_practice bodies, the four-stage path check they go through, and the size limits they are held to. Split from server.rs on the same terms as server/search.rs; every item that gained pub(super) did so because cargo check named it after a move made with no visibility changes at all.
grooveseek/src/server/kb_uri.rs (v1.0.0+) The corpus side of the kb:// resource surface: which documents this server will hand over, the URIs it advertises for them, and what a resources/read gets back. resources.rs below is the URI side — it builds kb:// strings and takes them apart, and knows nothing about the corpus; this is the half that knows it. Split from server.rs on the same terms as the other two.
grooveseek/src/prompts.rs (v0.22.0+) The MCP prompts surface: summarize_topic, deep_dive, whats_new, find_gaps, declared with rmcp’s #[prompt] / #[prompt_router] and dispatched by the #[prompt_handler] on server.rs’s ServerHandler. Each handler only calls the free function beneath it, so the tests exercise the production text without constructing a KbServer (which owns a Database and an Embedder) — the same split, for the same reason, as watcher::should_process_parts. Text-only messages: a prompt that embedded a resource would oblige the server to implement the resources capability as well. Fixed at compile time rather than configurable, because prompt text reaches the model and groove.toml is discovered, which would put a [prompts] section in the same privileged category as kb_path under restrict_untrusted. The router is generated with vis = "pub" so the handler in server.rs can reach it across the module boundary.
grooveseek/src/resources.rs (v0.22.0+) The MCP resources surface: the kb:// URI codec and the grouping behind resources/list, as pure functions with no database in sight so the encoding rules are testable directly. kb://topic/<prefix> names a topic group — the first one or two path segments, the same derivation the indexer uses for category / topic, so a group and the database agree without a second query to keep in step — and kb://doc/<path> names one document. Groups are built from KbCore::servable_document_paths() — the indexed paths minus those ServableRules will not hand over — and a read is checked against that same query, so a URI the listing offers cannot be refused when it is read back (ADR-0004). ServableRules (in server.rs) is the single predicate behind that query and the uri on a search hit: an extension the active parser registry can no longer open, or a recorded size past the cap a read would apply (ADR-0005). Separators stay / and everything else is percent-encoded; the traversal check runs after decoding, because %2e%2e%2f is ../ and a check that ran first would not see it. Tested against paths with non-ASCII characters, spaces and % even though the corpus this was built against contains none — an encoder that never fires is an encoder nobody has checked.
grooveseek/src/schema_compat.rs (v0.14.0+) Normalises the JSON Schema advertised for those tools before it leaves the server. schemars derives Option<T> as a union type ({"type": ["string", "null"]}) and stamps Rust integer widths as format: uint32; both are legal JSON Schema 2020-12, but strict tool-calling runtimes reject the union and do not know the format. Unrelated to schema.rs, which validates document frontmatter — the names are close, the subjects are not.
grooveseek/src/service/ (v0.8.0+) Cross-platform OS user service installer. mod.rs (= ServiceBackend trait + InstallContext + ServiceState), install.rs / uninstall.rs / status.rs (= orchestration), linux.rs / macos.rs / windows.rs (= per-OS backends, cfg-gated), plus two modules deliberately kept outside the cfg gates so they compile and are tested on every OS leg: render.rs (v0.14.0+, the unit / plist templates and their escaping — a plist bug used to be detectable only on the macOS runner) and powershell.rs (v0.14.0+, the UTF-8 output prelude and the strict / diagnostic decoders for what powershell.exe writes back). Phase 1 = user-level only (= no admin/sudo, Linux systemd-user / macOS LaunchAgent / Windows Task Scheduler AT_LOGON). groove service install self-registers using Rust crates only (= no NSSM / WiX / 3rd-party tooling). The Windows backend (v0.8.3+) invokes PowerShell’s Register-ScheduledTask -Action -Trigger -Settings cmdlet via Command::new("powershell") — schtasks /Create /XML was abandoned across v0.8.0 → v0.8.3 due to layered locale / elevation / Principal issues documented in .dev/knowledge/windows-task-scheduler-pitfalls.md.
grooveseek/src/indexer.rs walkdir-based file scan using Registry::extensions(). All read paths (initial scan, index_single_disk_entry, reindex_single_file, rename_single_file) go through links::read_checked (v0.20.0+; bytes, not read_to_string), which opens the file once and decides from that handle whether its bytes may be used before reading them — and hash the raw bytes with SHA-256 for content-hash diff detection; for existing UTF-8 KBs this is a no-op vs. the old string hash. Parses via the Parser trait (parse_bytes), embeds, stores. Per-file skip isolation: a read failure, a size-cap breach, or a parse_bytes error skips just that file (logged as a warning) instead of aborting the whole run, and the skipped path is retained in the index rather than pruned as deleted. A refusal (SingleResult::Refused) is kept distinct from those: it means the object opened was not the file that was collected, and rename_single_file maps it to its own outcomes rather than reporting a successful rename. Incremental APIs (reindex_single_file / deindex_single_file / rename_single_file) shared with the file watcher.
grooveseek/src/indexer/progress.rs (v0.7.8+) ProgressReporter + ProgressMode enum. Drives per-file output for groove index: Verbose (default) / Quiet (--quiet) / Auto (--progress, TTY = indicatif::ProgressBar, non-TTY = periodic Progress: N/M (P%) lines). MCP server rebuild_index tool wires Quiet directly. Bar lifetime is closed inside rebuild_index (lazy init via start_indexing(total)) so Backfilled / Found lines stay plain eprintln!.
grooveseek/src/parser/ Parser trait + Registry. mod.rs (Frontmatter / Chunk / ParsedDocument, plus the parse_bytes(bytes, path_hint, exclude_headings) -> Result<ParsedDocument> entry point that every call site — indexer and server — now goes through. parse_bytes lives on the ParserExt extension trait with a blanket impl<T: Parser + ?Sized>, so no parser can define it: it always delegates to Parser::parse_bytes_inner (same signature) inside a catch_unwind, and a panic anywhere in a parser or its dependencies becomes a per-file Err instead of aborting the whole index run. A default method would not do — it can be overridden, and an override silently bypasses the guard. parse_bytes_inner’s default impl validates UTF-8 then delegates to parse, so md/txt need no override, while binary-format parsers override it directly. is_binary() (default false) flags binary parsers for get_document’s size-cap classification and quality-filter exemption. MAX_RAW_BINARY_BYTES = 50 MiB, the shared raw-byte cap for binary formats used by both the indexer’s size-skip guard and get_document; MAX_RAW_TEXT_BYTES = 50 MiB does the same for text formats at index time since v0.17.0, so no format is read into memory unbounded), markdown.rs, txt.rs, pdf.rs (v0.10.0+, see below), ooxml.rs / xlsx.rs / docx.rs / pptx.rs (v0.11.0+, see below), panic_guard.rs (see below), registry.rs (extension lookup, binary_extensions()).
grooveseek/src/parser/code/ (v1.2.0+) Source code parsed into one chunk per definition, opt-in via [parsers].enabled = ["md", "rs"]. mod.rs holds the language-neutral half: LoadedGrammar (a grammar plus the tags query validated against it), CodeParser (one instance per extension, is_binary() == false so code is scored by the quality filter like prose), and the chunker. Definitions come from the grammar’s own tags.scm; a definition’s range extends backwards over an unbroken run of comment nodes above it, so a doc comment cannot land in both its definition’s chunk and the gap chunk above; an oversized definition recurses into nested definitions and splits by lines when it has none (the common path, since methods hold no nested definitions); every byte no definition covers becomes a heading-less gap chunk, with fragments under the quality filter’s short-content threshold dropped unless the file would otherwise produce nothing. Scope names come from walking Node::parent() for a name / type / trait field rather than from the definition tree, because Rust’s tags query captures impl blocks as references. symbol_kind carries the tags vocabulary verbatim (class covers struct / enum / union) rather than a language keyword, which is what keeps a grammar to data. MAX_RAW_CODE_BYTES = 1 MiB, compared with > so a file of exactly the cap parses as well as it reads — tree-sitter’s allocator aborts on OOM instead of unwinding, so panic_guard cannot catch it and the file has to be refused before the parser sees it. static_rust.rs holds the compiled-in Rust grammar behind the default-on grammar-rust feature. plugin.rs (v1.3.0+) is the other arrival path: it dlopens a library the user placed and checks it — exports present, groove’s own ABI version, the grammar’s tree-sitter ABI against the range this runtime speaks, the tags query compiling, and a single valid extension — before handing back the same LoadedGrammar. It opens a file named from a fixed id-to-library table rather than enumerating the directory, because opening a library runs its initialisers before any symbol can be inspected; an accepted library is then deliberately leaked, since the parse table and the tags query live inside it. See ADR-0012 and ADR-0013.
grooveseek/src/parser/panic_guard.rs The panic-isolation machinery behind ParserExt::parse_bytes: catch_parser_panic runs the parser under catch_unwind and turns a panic into Err("<path>: <id> parser panicked: <payload>"), keeping the payload so the indexer’s skip line still says what happened. A wrapper panic hook is installed once (never swapped) and consults a thread-local flag set by an RAII guard, so only the parsing thread’s own backtrace is suppressed while panics on other threads keep reporting — a hook that is swapped per call races when two threads parse at once. This started as PDF-only (v0.10.0) and moved here so docx/xlsx/pptx are covered too; without it a single crafted spreadsheet aborts groove index outright (calamine’s get_dimension subtracts unchecked, so ref="B2:A1" panics in any build with debug assertions).
grooveseek/src/parser/pdf.rs (v0.10.0+) PdfParser, is_binary() == true, opt-in via [parsers].enabled = ["md", "pdf"]. Extracts text page-by-page with oxidize-pdf (PdfReader + PdfDocument::extract_text), one non-empty page per chunk (heading p.N, level: None). Title / CreationDate PDF metadata become frontmatter, falling back to a filename-derived title when Title is absent. A malformed PDF that panics inside the crate’s parser internals degrades to a per-file Err (indexer skip-and-warn) instead of aborting the whole index run — that catch_unwind used to live here, and now sits in Parser::parse_bytes / parser/panic_guard.rs where every parser gets it. Extracted pages pass two gates, in this order (reject_unindexable_pages). First, mojibake detection (v0.15.1+), via two complementary signals: text whose C1 control codes (U+0080–U+009F) reach 1% of all characters is rejected — reading UTF-16BE one byte at a time lands the high bytes of U+8000–U+9FFF (and the low bytes of voiced kana) in C1, which correctly decoded text never contains, measured 0.00% across six correctly-extracted samples versus 3.61–15.59% across four mis-decoded ones — and text carrying the byte-pair signature of the same mis-decoding (≥30% of characters) is rejected, which covers the one shape that emits no C1 at all: unvoiced-kana-only text, whose high bytes are all 0x30, comes out as pure ASCII (あいうえお… → 0B0D0F…, 0.00% C1 at 407 chars/page, measured on oxidize-pdf 4.1.1 — the pin at the time; the pin since v0.15.2 is 4.3.0, which extracts this form correctly, leaving the gates as defense-in-depth). Long runs are judged per run (the leading parity near-constant, the other varied — the mirror orientation, alternating identifiers like 1A2A3A, is not produced by bytewise decoding and is not flagged); runs of two to seven characters — too short for the per-run judgment to be able to fire; a label sheet or word list splits into 4-char tokens (measured 148 chars/page) — are aggregated document-wide and judged as a pool. The mojibake gate runs first because mis-decoding turns one character into two, so mojibake clears the density gate below (1052 chars/page measured) while correctly extracted sparse text (29 chars/page) does not. Second, a PDF yielding under 50 chars/page on average is rejected. That heuristic was written for scanned/image-only PDFs (no text layer, no OCR support), but it is not specific to them: a PDF that decodes perfectly but genuinely carries little text per page — a cover sheet, a label, a figure-heavy deck — lands here too. The threshold is not lowered, because a scan carrying only digitally-added page numbers and a “CONFIDENTIAL” stamp measures 39 chars/page: a character count cannot separate worthless boilerplate from sparse real content. The diagnostic reports the measurement and offers common causes as an open list, rather than asserting “scanned” or enumerating a closed set. Japanese PDFs embedding a TrueType subset with /ToUnicode (Word / LibreOffice / Google Docs output) extract correctly; the historical mojibake case was a CID-keyed font with a predefined CMap and no /ToUnicode, whose root cause was upstream — oxidize-pdf read /DescendantFonts only when the CIDFont was an indirect reference, not when it was a direct dictionary. Reported and fixed by this project (bzsanti/oxidizePdf#469, shipped in oxidize-pdf 4.3.0, the pin since v0.15.2), so that form now extracts correctly too; the mojibake gates stay as defense-in-depth against decode failures from other causes. encrypted PDFs surface as an Err from PdfReader::new / extract_text (oxidize-pdf’s ParseResult-based error design, no password support). Post-processing joins conservative line-end hyphenation (-\n only when both neighbors are ASCII lowercase) and normalizes common ligatures (fi/fl/ff/ffi/ffl).
grooveseek/src/parser/ooxml.rs (v0.11.0+) Shared OOXML zip/XML helper consumed by xlsx.rs / docx.rs / pptx.rs (no parser struct of its own). read_zip_entry reads one zip part as raw bytes; core_xml_frontmatter / parse_core_xml map docProps/core.xml (Dublin Core: dc:title / dcterms:created or modified / cp:keywords) to Frontmatter, falling back to a filename-derived title when the part is missing or title is empty; local_name_pub strips a namespace prefix from a QName (cp:title → title) so element matching is prefix-agnostic; resolve_general_ref resolves quick-xml’s Event::GeneralRef (entity references such as &amp; arrive as a separate event, not folded into Event::Text; verified against the pinned 0.41) — factored out here because docx.rs and pptx.rs both need the same char-ref/named-entity handling.
grooveseek/src/parser/xlsx.rs (v0.11.0+) XlsxParser (.xlsx), is_binary() == true. XlsParser (.xls) still exists here but is not reachable from the registry since v0.14.0 (AU-06): calamine builds one dense cell grid per sheet while Xls::new runs, and BIFF bounds a sheet (65,536 x 256 = 512 MB of Data) but not a workbook, so a small file declaring many maximal sheets exhausts memory before groove regains control — and an allocation failure aborts the process instead of skipping the file. Bounding it would mean reading BOUNDSHEET / DIMENSIONS out of the CFB container ourselves, before Xls::new; deferred until someone asks for .xls. Both share one parse_workbook_bytes entry point that dispatches per format. Each opens the reader matching the registered extension (calamine::Xlsx / calamine::Xls) rather than probing formats, so a payload whose real container disagrees with its extension is rejected instead of parsed. .xlsx first runs a decompression preflight: every entry is checked against MAX_RAW_BINARY_BYTES both by its declared uncompressed size and by decompressing it for real (output discarded, bounded at the remaining budget + 1 byte), so the invariant needs no reference to naming — an archive may decompress to at most the cap, in total. The second layer is what stops a zip bomb, since the ZIP format does not enforce the declared size and zip 8.6 does not bound deflate output by it (measured: a 101 KB crafted workbook declaring 10 bytes expanded to 100 MB). Checking every entry rather than a suffix list is deliberate: calamine resolves parts through relationship Targets and ignores filenames, so xl/worksheets/payload is read like any worksheet — suffix-based selection was bypassed three times (fixed paths, then missing .rels, then the suffix assumption itself). The trade-off is that xl/media/ images count too, so a workbook near the raw cap that is mostly pictures can be skipped. Then one chunk per non-empty sheet (heading Sheet: <name>, tab-joined cell text per row), truncated at SHEET_MAX_BYTES (1 MiB) with row-aligned truncation semantics (the row that pushes the running total past the cap is still emitted whole, then extraction for that sheet stops — never cuts mid-row). Frontmatter comes from docProps/core.xml via ooxml::core_xml_frontmatter when the bytes open as a zip (true for .xlsx).
grooveseek/src/parser/docx.rs (v0.11.0+) DocxParser, is_binary() == true. Reads word/document.xml paragraph-by-paragraph (<w:p>), treating a <w:pStyle w:val="HeadingN"> as a section boundary — the same heading-hierarchy chunking rule markdown.rs uses for Markdown headings, including exclude_headings support (body under an excluded heading is dropped until the next non-excluded heading). Table (<w:tbl>) text needs no special-casing: the OOXML nesting w:tbl > w:tr > w:tc > w:p > w:r > w:t already funnels table cell text through the ordinary <w:p> boundary handling into the current section’s body. Frontmatter via ooxml::core_xml_frontmatter.
grooveseek/src/parser/pptx.rs (v0.11.0+) PptxParser, is_binary() == true. Collects ppt/slides/slideN.xml entries and sorts by the numeric slide index (not zip iteration order), emitting one chunk per slide (heading Slide N: <title> when a ctrTitle/title placeholder shape has text, else bare Slide N), with in-slide table text included in the body. Speaker notes are appended as a trailing [notes] section, resolved by reading the slide’s ppt/slides/_rels/slideN.xml.rels for a notesSlide relationship Target — deliberately not a same-numbered-file heuristic (slideN.xml ↔ notesSlideN.xml), which a dry-run (plan Task 3.7) showed can misattribute notes to the wrong slide when slide/notes numbering diverges after edits. Frontmatter via ooxml::core_xml_frontmatter.
grooveseek/src/doctor.rs (v0.23.0+) The checks behind groove doctor. Two groups. Integrity: whether chunks, vec_chunks and fts_chunks still agree about every chunk — a disagreement raises no error, it silently removes results from one half of the hybrid search, which is why backfill_fts exists at all. Servability: which indexed documents the resource surface withholds, computed by calling paths_with_unregistered_extension and ServableRules rather than reimplementing them, so the report cannot drift from what the server actually offers. It reports and never repairs — the contract paths_with_unregistered_extension already states — and each finding carries the command that fixes it. The SQL lives in db/meta.rs because Database::conn is private to that module, which also puts the queries next to backfill_fts, their nearest relative.
grooveseek/src/exclusion.rs (v0.21.0+) The one place that decides whether a knowledge-base path is out of scope, for all three surfaces that walk or watch it: the full index walk, the validate walk (which lives in the binary target and reaches this through the library’s public API) and the live watcher. ExclusionRules::load combines the built-in .git / .svn / node_modules fail-safe, exclude_dirs, and <kb_path>/.grooveignore — root only, no .gitignore, case_insensitive(true) set before any pattern is added because it does not apply retroactively. is_excluded(rel, is_dir) tests every ancestor as a directory first and stops at the first exclusion, which is what makes it agree with a walk that prunes: git’s rule is that a file under an ignored directory cannot be re-included with !. Only the matcher from the ignore crate is used and walkdir stays, because WalkBuilder defaults hidden() to true (on Windows that means dot-prefixed or FILE_ATTRIBUTE_HIDDEN), resolves add_ignore against the process cwd rather than the walk root, and makes .gitignore handling depend on require_git. matched_path_or_any_parents is deliberately not used: measured, it answers Whitelist for a file a walk would never reach, which would have put the drift back at the level of which API was called. The file itself is read through links::read_checked, capped at 64 KiB and 1000 patterns, with a leading BOM stripped; anything unreadable warns and is left out rather than stopping the run. The boundary is the index, not access — see validate_get_document_path.
grooveseek/src/links.rs (v0.19.0+) Hard-link detection for the three places a file can enter the index or leave it as content (the full index, the watcher, get_document), which already refuse symlinks. hard_link_count reads nlink from symlink_metadata on Unix and GetFileInformationByHandle on Windows — MetadataExt::number_of_links is still unstable there, and walkdir’s WIN32_FIND_DATAW metadata carries no link count — and is_multiply_linked fails open, since the check also gates deindexing and a deleted file has no link count. Checked after the extension filter at every call site: the Windows path needs the file opened, and on Linux every directory has a link count of at least two. (v0.20.0+) read_checked is the second entry point and the one the content actually passes through: it opens the file once and takes the link count, the file type and the size cap from that single fstat before reading the bytes from the same descriptor, so a hard link renamed over a collected path after the walk checked it is refused rather than read. On Unix the open adds O_NOFOLLOW and O_NONBLOCK (a symlink swapped in is refused; a FIFO cannot block the run); Windows adds neither, because symlink creation there needs administrator privilege and refusing reparse points would refuse OneDrive placeholders. The module doc states what the check does not do (link-then-unlink, intermediate-directory symlinks, and filesystems that always report a count of 1) and why the knowledge base directory is not a security boundary.
grooveseek/src/poison.rs (v0.19.0+) Recovering from a poisoned mutex instead of inheriting the panic. recover / recover_try take the LockResult rather than the mutex, so the literal .lock() stays at the call site where server’s meta-test scans for it; recover_db additionally rolls back a transaction a failed Drop-time ROLLBACK left open, since rusqlite swallows that error and every later &self write would join a transaction nobody commits. The first recovery in a process warns and names the mutex, later ones are debug — poisoning is sticky, so a per-recovery warning repeats for every request.
grooveseek/src/markdown.rs Thin shim over crate::parser::markdown::MarkdownParser, retained for legacy parse() / parse_with_excludes() callers.
grooveseek/src/watcher.rs notify-debouncer-full bridged to a tokio channel. Filters by extension and path — the path half through exclusion::ExclusionRules, the same object the index walk uses — then dispatches to indexer::{reindex,deindex,rename}_single_file. Runs alongside the MCP server via tokio::spawn. (v0.21.0+) A batch touching <kb_path>/.grooveignore rebuilds those rules before anything is classified, since the file has no registered extension and would otherwise be dropped by the same filter it is meant to change; run_watch_loop owns the state, so the reload needs &mut and no lock.
grooveseek/src/transport/ MCP transport abstraction. mod.rs (Transport enum + CLI/config resolution), stdio.rs (stdio), http.rs (rmcp StreamableHttpService + axum, mounts /mcp and /healthz; v0.8.0+ also mounts an admin sub-router with /ui + /api/admin/status, wrapped in admin_security_headers (CSP + nosniff, outermost so refusals carry them too). One dns_rebinding_gate serves every route that validates (ADR-0009): peer, then Host, then Origin, each group handed its own DnsRebindingGate state — /mcp the effective host and origin lists, the admin routes allowed_admin_hosts plus the same origin list plus the loopback-peer requirement, and /healthz the host list alone, and only when healthz_public = false (the default mounts it with no gate). rmcp’s own checks are given empty lists on purpose, so /mcp is validated here too — outside the session gate, i.e. before admission. /api/search was removed in v0.27.0; /ui searches through /mcp now). KbServerShared is Arc-shared through a session factory so each connection gets a lightweight handle.
grooveseek/src/transport/webui_index.html (v0.8.0+) The operator view served at /ui, embedded via include_str! in transport/http.rs::ui_index. Raw HTML + JS, no CSS framework, no external requests, XSS-safe via textContent / createElement only (= no innerHTML). A status band over a search box; the search goes through /mcp, so the page is also the smallest example of an MCP client over Streamable HTTP. Browsing a knowledge base is expected to move to MCP clients during 1.x — see stability.md.
crates/groove-tray/ (v0.9.0+) Windows-only system tray binary (groove-tray.exe, GUI subsystem) for daemon monitoring + lifecycle control. Polls /api/admin/status every 5s and renders a 4-state status dot (green / yellow indexing / red 1min+ down / gray polling-pending), right-click menu with 6 items (Status / Open Web UI / Start / Stop / Restart / Quit Tray). Start goes through PowerShell Start-ScheduledTask (= same path as grooveseek/src/service/windows.rs); stop does not — since v0.14.0 the tray reads the daemon’s pid from /api/admin/status and terminates that process through the Win32 API (OpenProcess once, then image-name check and TerminateProcess on the same handle, so a recycled pid cannot be hit). Stop-ScheduledTask still runs in both branches, but never as the deciding step: after a pid stop it is best-effort cleanup of the task instance a pre-v0.9.1 install would have (failures logged and ignored), and when the probe yields no usable pid — including an unreachable status endpoint — it is the fallback attempt whose error is deferred rather than returned. Either way success is decided by binding the daemon’s configured address, never by the mechanism’s own return value. Dual event loop: tao on the main thread, tokio runtime on a dedicated thread, bridged via EventLoopProxy::send_event. Panic hook + tracing-appender::rolling::daily write logs to %LOCALAPPDATA%\groove\logs\tray.YYYY-MM-DD. Library API (install::install_autostart / uninstall_autostart) is invoked by groove service install --with-tray / service uninstall / service tray-install / service tray-uninstall to manage the shell:startup .lnk shortcut via PowerShell WScript.Shell COM. cargo-dist publishes groove-tray.exe only for x86_64-pc-windows-msvc.
crates/groove-svc/ (v0.9.1+) Windows-only launcher binary (groove-svc.exe) that exists solely to start groove.exe serve without a console window. groove.exe is a console-subsystem binary, and Windows allocates a conhost window for it before the process runs, so hiding it in-process only removes it after a ~1 s flash (unfixed platform behavior, microsoft/terminal#249). This crate is windows_subsystem = "windows", so it has no console to pass on — but that alone is not enough: a console-subsystem child with no inheritable console calls AllocConsole() itself and gets a fresh visible window. The spawn therefore must pass CREATE_NO_WINDOW (0x0800_0000), which tells CreateProcess to skip console allocation for the child; that flag plus the windows-subsystem parent is what yields a 0-flash launch. It detach-spawns groove.exe with stdio nulled and exits. Since v0.9.1 the Task Scheduler Action points here instead of at groove.exe. child_args prepends serve unconditionally, which is why resolve_action_target in grooveseek/src/service/windows.rs leaves the Action’s -Argument clause empty — both halves of that invariant are unit-tested because a mismatch only surfaces at the next logon. Non-Windows builds compile to a fail-fast stub so the workspace builds everywhere; cargo-dist ships the binary only for x86_64-pc-windows-msvc. Consequence for the tray: the scheduled task’s own process exits immediately, which is why stopping the daemon cannot go through Stop-ScheduledTask (see the tray row).
crates/groove-grammar-abi/ (v1.2.0+) The contract between groove and a tree-sitter grammar. GrammarDescriptor carries the language name, the single file extension it claims, the parse table and the tags query; ABI_VERSION is groove’s own contract number, distinct from the grammar’s tree-sitter ABI, and is what a plugin declares so the loader can refuse a mismatch. The compiled-in Rust grammar builds a descriptor directly; a grammar shipped as a separate dynamic library builds the same one and exports it across a C ABI (the loader arrives in v1.3.0). Deliberately holds no logic — a grammar contributes data, and everything done with that data lives on groove’s side, which is what keeps adding a language from needing per-language code. #![forbid(unsafe_code)], publish = false, not shipped as a binary. See ADR-0013.
crates/groove-grammar-python/ (v1.3.0+) The Python grammar as a loadable library — the first grammar groove does not compile in. src/lib.rs is one invocation of groove_grammar_abi::groove_grammar_plugin!, naming the language (python), the extension it claims (py), and the parse table and tags query it takes from tree-sitter-python; what the C ABI needs lives in the macro, so a second grammar is a manifest and one macro call rather than an FFI surface to review again. Built as a cdylib, so cargo produces groove_grammar_python with the platform’s library decoration — the name parser/code/plugin.rs builds when it looks the file up. cargo-dist distributes it as its own app: dist = true alone is not enough for a package with no [[bin]], so the manifest also carries package-libraries = ["cdylib"] and cdylibs = [...], and it deliberately omits targets so the workspace’s four platforms are inherited rather than narrowed the way the two Windows-only binaries narrow theirs. The archive is named after the package, which is why the package name and the library name have to stay one name in cargo’s two spellings.
grooveseek/src/schema.rs Frontmatter schema validation. Reads groove-schema.toml under kb_path, enforces required / type / pattern / enum / min_length / max_length / allow_empty. Invoked by the groove validate CLI which reports in text / JSON / GitHub-annotation formats.
grooveseek/src/embedder.rs Thin wrapper over fastembed-rs. ModelChoice selects the embedding model (BGE-small-en-v1.5 / BGE-M3). RerankerChoice + Reranker provide optional cross-encoder reranking.
grooveseek/src/db.rs rusqlite + sqlite-vec + FTS5 (trigram). Manages the chunks / vec_chunks / fts_chunks schemas and CRUD. Exposes search_hybrid (Reciprocal Rank Fusion; the constant and the bm25 column weights are configurable via [search.fusion] since v0.13.0, defaults k = 60 and 2.0 / 1.0 / 1.0) and the v0.7.0 unbounded variants for the MMR / parent retriever pipeline. SearchFilters struct unifies filter args (path globs / tags / date range / min_quality); MatchSpan carries byte-offset citations (added in v0.3.0). chunks.level (added v0.7.0) distinguishes h2 / h3 headings.
grooveseek/src/db/schema.rs (v0.15.0+) Schema creation and forward migrations. Everything here runs from Database::init, which every constructor calls, so opening a database is what upgrades it.
grooveseek/src/db/search.rs (v0.15.0+) Retrieval: vector KNN, FTS5 candidates, and the RRF fusion that merges them. The half of the module whose behavior is observable as a number.
grooveseek/src/db/fts_query.rs (v0.16.0+) Compiles a query string into the FTS5 MATCH expression db/search.rs sends. parse_query keeps "..." regions verbatim (FTS5’s own doubled-quote convention), cuts the rest at separators and then at script boundaries (kanji / hiragana / katakana / other word characters), merges runs under the 3-character trigram floor into their neighbours within the same group, and joins the resulting phrases with ` OR ` (deduplicated, capped at 32). Every emitted phrase is a contiguous substring of the input — a trigram tokenizer matches nothing else, so a phrase that is not one could never match by construction. A query yielding no phrase falls back to the pre-v0.16.0 form (the whole trimmed query as one phrase) at 3 characters or more, and to vector-only below that. (v1.1.0+) A whitespace-delimited group led by - is tokenized the same way but excluded rather than searched for, capped separately at 32; ParsedQuery::match_expr joins the two sides as (positives) NOT (negatives), and ParsedQuery::positive_text is the raw query with the excluded groups cut out, for the embedder, the reranker, and match_spans (see ADR-0011). Query-side only: schema and tokenizer are unchanged, so no re-index is needed.
grooveseek/src/db/storage.rs (v0.15.0+) Documents and chunks. Writing a document is a multi-table operation — documents, chunks, fts_chunks, vec_chunks all have to agree — so several methods here open a transaction only when the caller has not (is_autocommit()).
grooveseek/src/db/meta.rs (v0.15.0+) Index-level metadata, statistics and whole-index maintenance: index_meta (embedding model / dim / context mode), document and chunk counts, path-to-hash maps for rename detection, and (AU-71) corpus_snapshot, which reads counts and a digest of the indexed chunks inside one transaction.
grooveseek/src/mmr.rs (v0.7.0+) Maximal Marginal Relevance greedy re-rank with a similarity cache. mmr_select operates on the post-rerank candidate pool and is gated by [search.mmr] config or the mmr per-call param.
grooveseek/src/parent.rs (v0.7.0+) Display-time parent retriever. apply_parent_retriever expands hit chunks via expand_adjacent (level-aware sibling merge) or expand_whole_document (full-doc fallback for chunks under whole_doc_threshold_tokens). Score / rank / match_spans stay on the original hit; only content and the new expanded_from field change.
grooveseek/src/quality.rs Per-chunk quality scoring (length / boilerplate / structure signals).
grooveseek/src/graph.rs Connection graph BFS over the vector index, for the get_connection_graph MCP tool and the groove graph CLI.
grooveseek/src/graph_render.rs Draws a finished connection graph: Graphviz DOT, and a standalone SVG laid out here rather than by a dependency (the walk produces a tree, so depth is the column and sibling order the row). CLI-only; the MCP tool stays JSON.
grooveseek/src/eval.rs Optional retrieval-quality evaluation for the groove eval CLI. Parses a golden YAML, runs each query through db.search_hybrid, and computes recall@k / MRR / nDCG@k. Loads / saves <kb_path>/.groove-eval-history.json for diff display. ConfigFingerprint (v0.7.0+) carries optional mmr / parent_retriever / fusion (v0.13.0+) so eval runs with different settings produce distinguishable history entries; each is recorded only when it differs from the built-in default, keeping older baselines comparable. Also scans the indexed corpus (v0.24.0+) for documents quoting two or more golden queries verbatim and reports them on stderr / in findings, without touching the exit code. Opt-in; does not affect serve / search / index.
grooveseek/src/tune.rs (v0.13.0+) Optional measurement tool for the groove tune CLI. Sweeps a fixed grid of RRF constants and FTS5 bm25 column weights over the golden query set, guards the result with nested leave-one-query-out CV (paired SE, selection stability, secondary-metric non-degradation — a sign test is also computed and reported, but decide does not gate on it), and prints either a paste-ready [search.fusion] snippet or a “keep the defaults” conclusion. Applies nothing automatically and never runs a reranker. Reuses eval’s GoldenSet / compute_query_metrics and db::fuse_rrf_ids.
grooveseek/src/tune/grid.rs (v0.15.0+) The parameter space groove tune sweeps and the per-query state carried through a sweep.
grooveseek/src/tune/stats.rs (v0.15.0+) The statistics behind an adopt / decline decision — mean, sample SD, paired SE, sign test, and the adoption thresholds ADOPT_MIN_MEAN_DELTA / ADOPT_SE_MULTIPLIER / STABILITY_MIN.
grooveseek/src/tune/report.rs (v0.15.0+) Rendering a sweep result for stdout (text via print!, plus the JSON form).
grooveseek/src/test_support.rs #[cfg(test)] only — the scratch directories the unit tests under src/** share, so that every one of them creates them the same way. A name is the process id, a nanosecond clock and an atomic counter: the first two collide between parallel threads of a single process, which was measured rather than assumed. A Drop guard removes the tree. The tempfile crate is deliberately not used. The integration tests under tests/ cannot reach this and carry their own copy in tests/common/temp.rs.

Data flow

.md / .txt / .pdf / .docx / .xlsx / .pptx / .rs files (.py needs a plugin)
(filtered by Registry::extensions(); only .md is on by default)
     │
     ▼ walkdir
indexer.rs: SHA-256 content-hash diff vs the chunks.hash column
     │
     ▼ changed files only
parser/: dispatch by extension → extract frontmatter + title + chunk
     │
     ▼
embedder.rs: embedding via fastembed
              (BGE-small-en-v1.5 → 384 dim, BGE-M3 → 1024 dim)
     │
     ▼
db.rs: UPSERT into chunks (metadata)
       + vec_chunks (embedding)
       + fts_chunks (FTS5 trigram)

At query time the search tool runs a hybrid:

The full v0.7.0 pipeline is RRF → reranker → MMR → parent retriever → match_spans. Each stage is a no-op when its config is off, so the pipeline collapses to pre-v0.7.0 behavior by default. See retrieval-pipeline.md for the narrative.

Contextual Retrieval (v0.12.0+)

Static Contextual Retrieval (feature-46) prepends a document-structure breadcrumb to each chunk before it reaches the embedder / FTS index / reranker, entirely at index time and with no LLM call. Gated by [contextual].enabled (default off as of v0.12.0 — a false-by-default judgment gate result, see “Contextual Retrieval” in docs/usage.md for the A/B numbers that drove it).

Embedding cache resolution

embedder.rs::resolve_cache_dir() picks in order:

  1. FASTEMBED_CACHE_DIR env var (highest priority)
  2. OS-standard cache directory joined with fastembed:
    • Linux: ~/.cache/fastembed
    • macOS: ~/Library/Caches/fastembed
    • Windows: %LOCALAPPDATA%\fastembed
  3. .fastembed_cache/ under CWD (final fallback)

First run downloads the chosen ONNX model to a HuggingFace-hub-compatible cache layout (BGE-small: ~130 MB, BGE-M3: ~2.3 GB, BGE-reranker-v2-m3: ~2.3 GB). Subsequent runs reuse the cache without re-downloading.

If fastembed-rs’s native TLS to HuggingFace fails (corporate proxies / TLS inspection), see “Working around HuggingFace TLS failures” in docs/clients.md for a huggingface_hub CLI workaround.

CLI output convention

The groove CLI follows a stdout = data, stderr = progress convention:

When writing subprocess tests, grep grooveseek/src/main.rs for the corresponding Commands::* block to confirm which channel each subcommand uses before asserting on the captured output — and remember the arm may delegate its printing to a helper (print_search_results / print_graph / print_validate_report / print_doctor_report). Read the two lists above rather than counting them: the split is settled in ADR-0010 and frozen by docs/stability.md, and a number written beside a list goes stale on its own. serve is a special case: it writes nothing as CLI output, but over the default stdio transport the MCP protocol itself occupies stdout, which is why a subprocess harness must keep draining it. Grep for print! as well as println! — the text branches of eval and tune use it.

Key dependencies