Full Text Index Search#44

The current API filters are quite handy, but lack the ability to support powerful searching of large texts. I’d like to see the ability to define custom search endpoints/fields that provide scored, elastic-like search results. The ultimate goal is to to give customers the ability to build Google-like SERPs on top on their GraphCMS models.

I’d imagine a new type, called Index, where I can:

• Provide a name (which would produce a query field in my API)
• Choose which types are available in the index (would be nice to have optional filters here for security/performance)
• Decide which fields of each types should go into the index, and how (perhaps only allow fields with a native text output)
• Perhaps see some metrics for size, use, status, and billing impact of each index I create

This has come up in conversation on Slack, which yielded a few options for how to go about building it:

1. Full query field access to a single type. Simpler to implement, allows for single requests to also return the fields you want, but extremely limited (all but the simplest use cases require searching across types).
2. Limited searching across multiple types, only returning guaranteed common fields, like type and id. This gives you searching across types, with the added cost of additional searches to pull back the fields you need.
3. Full searching across multiple types, returning fragments for each type contained within the index. By far the most powerful option, but requires support for a union type. See this Prisma issue.
4. Full search across multiple types, returning common fields, plus some faked/mapped preview content (maybe the index builder UX lets you map which fields from the source types get mapped to a generic SearchResult type).

Option 1 seems doable, but so limited as to be pointess (I’d just stand up my own search service and syndicate GraphCMS data to it). Same for Option 2.

Option 3 is the pie-in-the-sky option, but requires Prisma to support union types, which they’ve been pretty quiet about recently.

Option 4 might be the best hybrid approach. Essentially, when you define an index, you’re also defining a new model to represent the results. When you add a new type to the index, you choose which fields from that model map to the fields in your index (e.g., title, description, date, preview image, etc.). That way you have a consistent interface that doesn’t require a union type. They could also provide their original type, if you wanted to fetch more fields in subsequent calls.

Having thought about it some more, I think some flavor of option 4 would be both the best and easiest approach. Defining a CMS type for each index is like a type in elastic, and mapping fields of a GraphCMS type when you add it to a search index would provide all you need to map fields in the underlying elastic indexes, if you were to build it on elastic. The UX for that wouldn’t be bad at all, and could remain quite consistent with the existing GraphCMS flows.

I know some of the implementation specifics have changed in recent versions of elastic, but the principles are the same.

any updates on this ?