Honcho

Honcho is a content engine that indexes large collections of documents—research archives, institutional publications, news corpora, internal knowledge bases—and makes them searchable and accessible to AI assistants. Import tens of thousands of articles from a CMS, bulk-load a document archive, or crawl hundreds of feeds over time. Everything lands in the same full-text index.

What makes it powerful is what happens next. Connect Honcho to Claude or another AI assistant, and the entire corpus becomes conversational. Ask “how have expert views on China’s economy evolved over the past decade?” and the AI synthesizes across thousands of indexed documents—threading together analysis, tracking how positions shifted, surfacing connections that no keyword search would find.

Content flows in from many directions: bulk importers for CMS migrations and document archives, an extract API with declarative rules for structured data sources, RSS/Atom crawling for ongoing feeds, and AI assistants that save articles and notes on your behalf.

It supports multiple users and groups, so a team can share a common pool of content and collaborate through their AI assistants. One person’s assistant can summarize something and share it to the team’s feed, building a shared knowledge base without anyone copying and pasting links around.

Honcho is also a persistence layer for AI workflows. Agent apps that summarize articles, monitor topics, or produce analysis can write their output back to Honcho, where it becomes searchable alongside everything else. The content you curate and the content your tools produce all live in one place.

• Aggregate Crawl RSS, Atom, and HTML sources on configurable schedules. A declarative rules engine lets you define per-site extraction logic—CSS selectors, field mappings, timestamp parsing, fallback chains—in a config file instead of code. Built-in deduplication and change detection keep the index clean.
• Ingest Content doesn’t just come from feeds. AI assistants save articles and notes via MCP. A save-URL API endpoint supports bookmarklets and mobile shortcuts. An extract API accepts raw JSON, HTML, or XML and runs it through extraction rules. Bulk importers handle WordPress exports and other CMS migrations. Everything lands in the same index.
• Index Full-text search powered by Lucene 9 with a configurable text analysis pipeline. Boolean queries, time-range filters, tag and topic facets, custom numeric fields, and configurable relevance scoring.
• Store Persistent content storage with content hashing for change detection. Encrypted backups with cloud KMS integration. Protobuf-based metadata model.
• Serve GraphQL, REST, and MCP (Model Context Protocol) endpoints for search, retrieval, and content distribution. No rendering opinions—bring your own frontend, feed reader, mobile app, or AI assistant.
• Replicate Push and pull replication between instances. Distribute content across organizational boundaries with topic-based routing and editorial workflows.

• Search is built in, not bolted on The Lucene index is the primary read path. Content is indexed at write time with no sync lag. The query API exposes the full power of Lucene: phrase queries, field-scoped search, boolean composition, range-boosted relevance, and custom numeric fields for domain-specific ranking.
• Structured content fragments Content is modeled as typed fragments—paragraphs, code blocks, headings, pull quotes, recipe steps. Each fragment type is indexed separately, so you can search within specific block types: find all entries where a code block mentions HashMap, or where a heading contains authentication.
• Declarative content extraction A rules DSL lets you define how to extract entries from any JSON, HTML, or XML source—selectors, extractors, transforms, timestamp parsers—without writing code. Rules compose via config layering: define a generic RSS base, then override only the fields that differ per source. You can develop rules conversationally through an AI assistant—paste in your raw content, iterate on the rules until the extraction is right, then save them.
• One system, not six Each service in the typical stack is another deployment, another set of credentials, another failure mode, another thing to keep in sync. Honcho’s tight integration eliminates the boundaries where things break.

• Runtime Java 17 on Jetty 12 with Jakarta Servlet.
• Search engine Lucene 9 with unified numeric fields, near-real-time search with searcher warm-up, and proper FILTER vs MUST clause handling.
• API layer GraphQL for flexible queries, REST for JSON/RSS/Sitemap output, and MCP (Model Context Protocol) for AI assistant integration.
• Data model Protocol Buffers for internal data model and wire format, with a purpose-built JSON encoder for browser and API clients.
• Instrumentation Dropwizard Metrics on every significant operation—search latency, indexing throughput, crawl rates, storage I/O.
• Text analysis Pluggable stemmer (KStem, Porter, minimal, none), stop words, protected words, ASCII folding, per-index analyzer selection.
• Custom fields Define domain-specific indexed fields via configuration with configurable tokenization and case handling.

• Read and write AI assistants can search and retrieve content, but also create entries, update metadata, manage tags, and curate the index—all through the same MCP interface.
• Structured retrieval Content fragments let MCP tools return specific block types—code examples, definitions, key paragraphs—rather than dumping entire documents into the context window.
• Multi-source aggregation Hundreds of curated sources through one interface—industry news, internal docs, regulatory updates—without the model needing to know where each piece lives.
• Real-time content Continuous crawling and near-real-time indexing. Content is searchable within seconds of arrival.
• Group collaboration Multiple users sharing an MCP server can write to shared group feeds. “Summarize this and send it to the research group”—an AI-mediated collaboration channel where each person's assistant contributes to a shared knowledge pool.
• AI memory Assistants can save and recall notes across conversations. “Remember that the client prefers weekly reports on Mondays”—and it’s there next time you ask.
• Messaging Lightweight messaging through group feeds. AI assistants can send messages, share summaries, and post updates on behalf of their users.
• Cross-posting Share entries across groups while preserving original author attribution. Add commentary when sharing—provenance metadata tracks who shared what and from where.
• Personalized digest Users build a personal digest by favoriting authors, sources, hosts, and search queries. “Give me my morning briefing” returns a curated feed of what matters to them, updated in real time.
• Feed management AI assistants can discover feeds from any URL and add them as crawled sources—“follow this site” becomes a single conversational command.
• Multi-user isolation Each authenticated user sees only their assigned sources. OAuth with DB-backed tokens provides secure, persistent access.

• Knowledge base for AI Aggregate domain-specific content, index with structured fragments, and serve to AI assistants via the built-in MCP server. AI assistants can search, create content, manage sources, and collaborate through groups—the entire index becomes conversational.
• Content aggregation and curation Crawl and normalize hundreds of sources—industry publications, competitor blogs, wire services, regulatory feeds—into a single searchable interface with editorial taxonomy and workflows.
• Headless search API Lucene-quality full-text search as a service. Boolean queries, faceted filtering, configurable relevance, time-range constraints, and custom ranking signals—without the operational overhead of Elasticsearch or per-query cost of hosted search.

• Feed infrastructure Crawl RSS/Atom/HTML, normalize to a clean structured model, re-encode as JSON or protobuf. Per-source extraction rules handle non-standard feeds and custom HTML without code changes—deduplication, metadata enrichment, and configurable update intervals.
• Content archiving Long-term content storage with full-text search and retrieval. Encrypted backups for compliance and retention requirements.