Throughout summer season 2021, I bought fortunate sufficient to cross Twitter paths with Peter Szinek, who launched me to his workforce and bought me employed at RCRDSHP, Obie Fernandez’s newest Net 3.0 mission involving Music NFTs. Being myself a professional musician and music producer, I used to be thrilled to lastly be capable to combine my two passions-turned-into-a-living. Surrounded by superior builders, I realized and constructed tons of cool stuff, amongst which a reactive, tremendous performant server-side-rendered search expertise utilizing StimulusReflex, ElasticSearch and near no JavaScript. The function is still live and just about unchanged.
The aim of this sequence will likely be to first reimplement this tremendous pleasant UX with primary filters and kind choices, together with StimulusReflex. Then, we’ll see how ElasticSearch can enable extra advanced filters and search situations, whereas bettering efficiency. Within the final episode, we’ll try to substitute StimulusReflex with the brand new Customized Turbo Stream Actions and examine implementation/behaviour. If time permits, I would add a bonus episode to point out the way to deploy all this in manufacturing. Let’s dig in.
What are we constructing?
Keep in mind how again within the day, individuals used to gather artwork printed on precise paper ? A bit like NFTs, solely bodily. Bizarre, proper? Nicely, let’s image an app that may enable customers to purchase and promote restricted version artwork prints. The prints would come with images, film posters, and illustrations of varied codecs. It ought to seem like that:
Right here’s what the DB appears like:
Please be aware the tags column is of string array sort. One may argue that the Itemizing desk, in our case, might simply be skipped. However for the sake of holding a real-world complexity state of affairs, let’s say that we’d prefer to preserve the precise Prints separate from their listings (and for the reason that app permits customers to promote their prints, we’d moderately suppose {that a} print may very well be listed a number of instances).
OK. Present me the gear
First issues first: on the frontend, we’ll use StimulusReflex (a.ok.a SR) to construct a brilliant reactive and pleasant search expertise with little or no code, and little to no JavaScript. For these unfamiliar:
StimulusReflex is a library that extends the capabilities of each Rails and Stimulus by intercepting consumer interactions and passing them to Rails over real-time websockets. The present web page is shortly re-rendered and morphed to replicate the brand new utility state.
Sounds a bit like Hotwire on paper, although you’ll see how their philosophy drastically differs within the final episode of this sequence. We’ll additionally use a sprinkle of CableReady, an in depth cousin of SR.
On the backend, we’ll want just a few instruments. Other than the classics (ActiveRecord scopes and the pg_search gem), you’ll see how the (but formally unreleased however production-tested) all_futures gem, constructed by SR authors, will act as a perfect ephemeral object to quickly retailer our filter params and host our search logic. Lastly, we’ll use pagy for pagination duties.
Philtre d’amour
(Please indulge this shitty French pun, the expression philtre d’amour that means love potion but additionally feels like beloved filter)
Let’s begin by creating some easy information. We’ll add just a few artworks of various type : images, illustrations, and posters. Every may have a number of tags from a given checklist and an creator. For now, let’s simply generate one print per art work, and an inventory for every. Prints could be considered one of 3 accessible codecs , whereas listings will likely be of various value.
Now let’s checklist our completely different options:
- Search by identify or creator
- Filter by minimal value
- Filter by most value
- Filter by class
- Filter by print format
- Filter by tags
- Order by value
- Order by date listed
Earlier than I began constructing my function, my former colleague and pal @marcoroth pointed me to leastbad’s Beast Mode, from which I took heavy inspiration to get going. That’s how I found his gem all_futures, which offers us with an ActiveRecord-like object that can persist to Redis. Let’s see how issues seem like.
# app/controllers/listings_controller.rb
class ListingsController < ApplicationController
def index
@filter ||= ListingFilter.create
@listings = @filter.outcomes
finish
finish
# app/fashions/listing_filter.rb
class ListingFilter < AllFutures::Base
# Filters
attribute :question, :string
attribute :min_price, :integer, default: 1
attribute :max_price, :integer, default: 1000
attribute :class, :string, array: true, default: []
attribute :tags, :string, array: true, default: []
attribute :format, :string, array: true, default: []
# Sorting
attribute :order, :string, default: "created_at"
attribute :route, :string, default: "desc"
def outcomes
# TODO: Construct a question out of those attributes
finish
finish
Discover how params are absent from the controller, and nothing will get handed to our ListingFilter object? And the way come @filter might doubtlessly be already outlined? You’ll see why in a bit, so let’s first have a look at constructing the question.
In his method, @leastbad merely created an ActiveRecord scope for every filter, then very cleverly and neatly, chained them to construct his ultimate filtered question, very like this:
# In app/fashions/listing_filter.rb
def outcomes
Itemizing.for_sale
.price_between(min_price, max_price)
.from_categories(class)
.with_tags(tags)
.with_formats(format)
.search(question)
.order(order => route)
finish
You may surprise: “However what if filters are empty and arguments clean? The chain’s gonna break!”. Nicely, take a look on the scopes declaration:
# app/fashions/itemizing.rb
class Itemizing < ApplicationRecord
belongs_to :print
scope :for_sale, ->{ the place(sold_at: nil) }
scope :price_between, ->(min, max) { the place(value: min..max) }
scope :with_formats, ->(format_options) { joins(:print).the place(prints: {format: format_options}) if format_options.current? }
scope :from_categories, ->(cat_options) { joins(:art work).the place(artworks: {class: cat_options}) if cat_options.current? }
scope :with_tags, ->(choices) { joins(:art work).the place("artworks.tags && ?", "{#{choices.be part of(",")}}") if choices.current? }
scope :search ->(question) { # TODO }
finish
In ListingFilter, the essential bit is to be sure that each attribute has a default worth. The magic then happens within the if assertion on the finish of the scopes anticipating an argument: if the lambda returns nil, then it should basically be ignored, and the gathering returned as is. Such a pleasant trick. Time for some specs to make sure that issues really work:
RSpec.describe ListingFilter, sort: :mannequin do
let!(:picture) { Paintings.create(identify: "Canines", creator: "Elliott Erwitt", 12 months: 1962, tags: %w[Animals B&W USA], class: "pictures") }
let!(:poster) { Paintings.create(identify: "Fargo", creator: "Matt Taylor", 12 months: 2021, tags: %w[Cinema USA], class: "poster") }
let!(:photo_print) { picture.prints.create(format: "30x40", serial_number: 1) }
let!(:photo_print_2) { picture.prints.create(format: "18x24", serial_number: 200) }
let!(:poster_print) { poster.prints.create(format: "40x50", serial_number: 99) }
let!(:photo_listing) { photo_print.listings.create(value: 800) }
let!(:photo_listing_2) { photo_print_2.listings.create(value: 400) }
let!(:poster_listing) { poster_print.listings.create(value: 200) }
let!(:sold_listing) { poster_print.listings.create(value: 300, sold_at: 2.days.in the past) }
describe "#outcomes" do
it "does not return a bought itemizing" do
anticipate(ListingFilter.create.outcomes).not_to embody(sold_listing)
finish
context "Filter choices" do
it "Filters by value" do
filter = ListingFilter.create(min_price: 100, max_price: 300)
anticipate(filter.outcomes).to match_array([poster_listing])
filter.replace(max_price: 1000)
anticipate(filter.outcomes).to match_array([photo_listing, photo_listing_2, poster_listing])
finish
it "Filters by format" do
filter = ListingFilter.create(format: ["40x50"])
anticipate(filter.outcomes).to match_array([poster_listing])
filter.replace(format: ["40x50", "30x40"])
anticipate(filter.outcomes).to match_array([photo_listing, poster_listing])
finish
it "Filters by class" do
filter = ListingFilter.create(class: ["photography"])
anticipate(filter.outcomes).to match_array([photo_listing_2, photo_listing])
filter.replace(class: ["photography", "poster"])
anticipate(filter.outcomes).to match_array([photo_listing, photo_listing_2, poster_listing])
finish
it "Filters by tags" do
filter = ListingFilter.create(tags: ["Cinema"])
anticipate(filter.outcomes).to match_array([poster_listing])
filter.replace(tags: ["Cinema", "Animals"])
anticipate(filter.outcomes).to match_array([photo_listing, photo_listing_2, poster_listing])
finish
it "Filters by a number of attributes" do
filter = ListingFilter.create(tags: ["Cinema"], max_price: 300, class: ["poster"])
anticipate(filter.outcomes).to match_array([poster_listing])
finish
finish
finish
finish
All inexperienced. I hope you’ll respect how simple it’s to check this. Let’s shortly add pg_search to our Gemfile, then deal with the search scope:
# app/fashions/itemizing.rb
class Itemizing < ApplicationRecord
embody PgSearch::Mannequin
belongs_to :print
has_one :art work, via: :print
scope :search, ->(question) { basic_search(question) if question.current? }
# Skipping the opposite scopes...
pg_search_scope :basic_search,
associated_against: {
art work: [:name, :author]
},
utilizing: {
tsearch: {prefix: true}
}
#...
finish
Since our listings don’t carry a lot data, we’ll have to leap just a few tables to look the place we’d like, particularly the identify and creator columns of our Paintings mannequin. Sadly, pg_search doesn’t assist related queries additional than 1 desk away, thus the has_one... via relationship we would have liked so as to add. Let’s add some assessments for the search:
context "Search" do
it "renders all listings if no question is handed" do
filter = ListingFilter.create(question: "")
anticipate(filter.outcomes).to match_array([photo_listing, photo_listing_2, poster_listing])
finish
it "can search by art work identify or creator" do
filter = ListingFilter.create(question: "Erwitt")
anticipate(filter.outcomes).to match_array([photo_listing, photo_listing_2])
filter.replace(question: "Fargo")
anticipate(filter.outcomes).to match_array([poster_listing])
finish
it "can each search and kind" do
filter = ListingFilter.create(question: "Erwitt", order_by: "value", route: "asc")
anticipate(filter.outcomes.to_a).to eq([photo_listing_2, photo_listing])
filter.replace(question: "Erwitt", order_by: "value", route: "desc")
anticipate(filter.outcomes.to_a).to eq([photo_listing, photo_listing_2])
finish
finish
We run the assessments and naturally, all the things is gr… Oh no. Seems just like the final check is performing up:
Apparently, a known problem of pg_search is that it doesn’t play effectively with keen loading, nor combos of be part of and the place queries. The beneficial workaround (and my standard plan B when ActiveRecord queries begin to get ugly) is to make use of a subquery:
# In app/fashions/listing_filter.rb
def outcomes
filtered_listings_ids = Itemizing.for_sale
.price_between(min_price, max_price)
.from_categories(class)
.with_tags(tags)
.with_formats(format)
.pluck(:id)
Itemizing.the place(id: filtered_listings_ids)
.search(question)
.order(order_by => route)
.restrict(200)
finish
Let’s additionally add some final specs for the kind choices and run all this.
context "Kind choices" do
specify "Current listings first (default behaviour)" do
filter = ListingFilter.create
anticipate(filter.outcomes.to_a).to eq([poster_listing, photo_listing_2, photo_listing])
finish
specify "Most costly first" do
filter = ListingFilter.create(order_by: "value", route: "desc")
anticipate(filter.outcomes.to_a).to eq([photo_listing, photo_listing_2, poster_listing])
finish
specify "Least costly first" do
filter = ListingFilter.create(order_by: "value", route: "asc")
anticipate(filter.outcomes.to_a).to eq([poster_listing, photo_listing_2, photo_listing])
finish
finish
All the pieces’s inexperienced… Apart from the search and filter choice. The error’s gone, however the check nonetheless fails; the ordering doesn’t appear to work, regardless of all of the sorting assessments being inexperienced. After one other lookup on pg_search recognized points, it seems that order statements following the search scope don’t work. Workarounds embody utilizing reorder as a substitute, or transferring the order clause up the chain. I opted for the primary choice, which make all assessments move. Let’s transfer on.
Stairway to Heaven
Now that we all know that our backend is working because it ought to, let’s wire up our stuff. I’m gonna skip on Stimulus Reflex setup and configuration and dive proper in. You possibly can simply comply with the official setup or, when you use import-maps, comply with @julianrubisch’s article on the subject. I additionally know that leastbad has been engaged on an automatic installer that detects your configuration and units all the things up for you when you care to strive it earlier than the following model of SR will get launched.
When you’re completed with that, let’s start with the kind first. Let’s recap our sorting choices and retailer them someplace:
class ListingFilter < AllFutures::Base
SORTING_OPTIONS = [
{column: "created_at", direction: "desc", text: "Recently added"},
{column: "price", direction: "asc", text: "Price: Low to High"},
{column: "price", direction: "desc", text: "Price: High to Low"}
]
#...
attribute :order_by, :string, default: "created_at"
attribute :route, :string, default: "desc"
#...
# Memoizing the worth to keep away from re-computing at each name
def selected_sorting_option
@_selected_option ||= SORTING_OPTIONS.discover choice
finish
finish
Then in our “Kind by” dropdown, we’ll have one thing like :
<div class="dropdown">
<button>
Kind by:<span><%= @filter.selected_sorting_option[:text] %></span>
</button>
<!-- Skipping numerous HTML -->
<% ListingFilter::SORTING_OPTIONS.every do |choice| %>
<% if choice == @filter.selected_sorting_option %>
<span class="font-semi-bold ..."><%= choice[:text] %></span>
<% else %>
<a data-reflex="click->Itemizing#kind"
data-column="<%= choice[:column] %>"
data-direction="<%= choice[:direction] %>"
data-filter-id="<%= @filter.id %>"
href="#"
>
<%= choice[:text] %>
</a>
<% finish %>
<% finish %>
</div>
Even when you’re unfamiliar with StimulusReflex, it ought to nonetheless remind you of the way in which we invoke common stimulus controllers. Solely right here, when our hyperlink will get clicked, it ought to set off the kind motion (a ruby technique) from the Itemizing reflex (a ruby class). Let’s code it:
# app/reflexes/listing_reflex.rb
class ListingReflex < ApplicationReflex
def kind
@filter = ListingFilter.discover(ingredient.dataset.filter_id)
@filter.order_by = ingredient.dataset.column
@filter.route = ingredient.dataset.route
@filter.save
finish
finish
And certain sufficient, it really works! So what is going on on right here? Nicely, clicking the hyperlink invokes our reflex, which will get executed proper earlier than our present controller motion runs once more. It permits us to execute any type of server-side logic, in addition to play with the DOM in varied methods, however with ruby code. Then, the DOM will get morphed over the wire.
What we did in our particular case: since our filter object is being persevered in Redis, it has a public id, which we saved as a data-attribute, and later retrieved from our reflex motion. Then, we fetched the item from reminiscence and up to date it with new attributes. Because of this @filter will likely be already outlined by the point we get to that time. By default, not specifying something extra in our motion will trigger SR to simply re-render the entire web page earlier than operating the controller motion. We may very well be extra particular right here, and simply select to morph just a few components to avoid wasting treasured milliseconds. However for demo functions we’ll go away it as is.
Let’s add a filter subsequent. We’ll begin with the primary one, by minimal value.
<div class="text-sm text-gray-600 flex justify-between">
<label for="min-price">Minimal Value:</label>
<span><output id="minPrice">50</output> $</span>
</div>
<enter sort="vary"
data-reflex="change->Itemizing#min_price"
data-filter-id="<%= @filter.id %>"
identify="min-price"
min="50"
max="1000"
worth="<%= @filter.min_price %>"
class="accent-indigo-600"
oninput="doc.getElementById('minPrice').worth = this.worth"
>
I bought lazy and didn’t wish to code an additional stimulus controller simply to point out the value worth. However aside from that, we simply want so as to add the brand new #min_price motion:
# app/reflexes/listing_reflex.rb
class ListingReflex < ApplicationReflex
def kind
@filter = ListingFilter.discover(ingredient.dataset.filter_id)
@filter.order_by = ingredient.dataset.column
@filter.route = ingredient.dataset.route
@filter.save
finish
def min_price
@filter = ListingFilter.discover(ingredient.dataset.filter_id)
@filter.min_price = ingredient.dataset.worth
@filter.save
finish
finish
And right here in motion:
I believe by now you get the image. Let’s simply do the search and one of many checkbox filters.
Within the view:
<!-- Search -->
<enter sort="search" worth="<%= @filter.question %>" data-filter-id="<%= @filter.id %>" data-reflex="change->Itemizing#search">
<!-- Format Filter -->
<% Print::FORMATS.each_with_index do |format, index| %>
<div class="flex items-center">
<enter data-reflex="change->Itemizing#format" <%= "checked" if @filter.format.embody? format %> data-filter_id="<%= @filter.id %>" worth="<%= format %>" sort="checkbox">
</div>
<% finish %>
Our Reflex actions are beginning to be fairly related to one another, which requires a refactor. You possibly can’t do any higher than leastbad’s method, particularly when you begin having extra sophisticated logic happening (like customized morphs or pagination):
# app/reflexes/listing_reflex.rb
class ListingReflex < ApplicationReflex
def kind
update_listing_filter do |filter|
filter.order_by = ingredient.dataset.column
filter.route = ingredient.dataset.route
finish
finish
def min_price
update_listing_filter do |filter|
filter.min_price = ingredient.worth.to_i
finish
finish
def max_price
update_listing_filter do |filter|
filter.max_price = ingredient.worth.to_i
finish
finish
def format
update_listing_filter do |filter|
filter.format = ingredient.worth
finish
finish
def search
update_listing_filter do |filter|
filter.question = ingredient.worth
finish
finish
personal
def update_listing_filter
@filter = ListingFilter.discover(ingredient.dataset.filter_id)
yield @filter
@filter.save
# Add customized morphs right here or any logic earlier than the controller motion is run
finish
finish
And so forth with the opposite filters. We will now mix search, filters and kind choices with no web page refresh.
Ambrosia on the cake
Let’s improve the UX a bit. Proper now there’s no pagination. Straight after including pagy, clicking any web page hyperlink will navigate, inflicting the params to reset. Let’s repair this by overriding pagy’s default template and wire hyperlinks to our Reflex as a substitute:
<!-- views/listings/_pagy_nav.html.erb -->
<% hyperlink = pagy_link_proc(pagy) -%>
<%# -%><nav class="pagy_nav pagination space-x-4" position="navigation">
<% if pagy.prev -%> <span class="web page prev"><a class="text-indigo-400" href="#" data-reflex="click->Itemizing#paginate" data-filter-id="<%= filter.id %>" data-page="<%= pagy.prev || 1 %>">Earlier</a></span>
<% else -%> <span class="web page prev text-gray-300">Earlier</span>
<% finish -%>
<% pagy.sequence.every do |merchandise| -%>
<% if merchandise.is_a?(Integer) -%> <span class="web page"><a class="text-indigo-400 hover:text-indigo-600" href="#" data-reflex="click->Itemizing#paginate" data-filter-id="<%= filter.id %>" data-page="<%= merchandise %>"><%== merchandise %></a></span>
<% elsif merchandise.is_a?(String) -%> <span class="web page page-current font-bold"><%= merchandise %></span>
<% elsif merchandise == :hole -%> <span class="web page text-gray-400"><%== pagy_t('pagy.nav.hole') %></span>
<% finish -%>
<% finish -%>
<% if pagy.subsequent -%> <span class="web page subsequent"><a class="text-indigo-400 hover:text-indigo-600" href="#" data-reflex="click->Itemizing#paginate" data-filter-id="<%= filter.id %>" data-page="<%= pagy.subsequent || pagy.final %>">Subsequent</a></span>
<% else -%> <span class="web page subsequent disabled">Subsequent</span>
<% finish -%>
<%# -%></nav>
# Add the paginate technique to ListingReflex
def paginate
update_listing_filter do |filter|
filter.web page = ingredient.dataset.web page
finish
finish
# And replace the controller
def index
@filter ||= ListingFilter.create
@pagy, @listings = pagy(@filter.outcomes, objects: 12, web page: @filter.web page, dimension: [1,1,1,1])
# Can generally occur over navigation when assortment will get modified in actual time
rescue Pagy::OverflowError
@pagy, @listings = pagy(@filter.outcomes, objects: 12, web page: 1, dimension: [1,1,1,1])
finish
One other subject is that for the time being, updating filters and our sorting choices don’t replace the URL params; refreshing the web page clears all the things, and we’re not capable of save or share the results of our search to somebody. Let’s deal with that as effectively. What we wish is for our URL to all the time replicate the present state of filters on one hand, then be capable to load our filter params from the URL then again.
First step is made simple by the mighty cable_ready library, particularly its push_state operation. Not solely is it nearly magical, but it surely is able to use in any Reflex. Take a look at all you can do with it. Here’s what our most important motion must do what we wish:
# reflexes/listing_reflex.rb
def update_listing_filter
@filter = ListingFilter.discover(ingredient.dataset.filter_id)
yield @filter
@filter.save
# Updating URL with serialized attributes from our filter
cable_ready.push_state(url: "#{request.path}?#{@filter.attributes.to_query}")
finish
Now when you change any filter, sort any question, change web page or change sorting choice, the URL will replace itself, together with each filter attribute. Our final step is to load these attributes from the params on preliminary web page load:
class ListingsController < ApplicationController
embody Pagy::Backend
def index
@filter ||= ListingFilter.create(filter_params)
@pagy, @listings = pagy(@filter.outcomes, objects: 12, web page: @filter.web page, dimension: [1,1,1,1])
rescue Pagy::OverflowError
@pagy, @listings = pagy(@filter.outcomes, objects: 12, web page: 1, dimension: [1,1,1,1])
finish
personal
# Remember to replace this checklist when including filter choices
def filter_params
params.allow(
:question,
:min_price,
:max_price,
:web page,
:order_by,
:route,
class: [],
tags: [],
format: []
)
finish
finish
It’s beginning to get fairly nifty. One final subject UX-wise : since we are able to not refresh to clear all of it, we lack a Clear All button. Simply add a hyperlink, then wire it to a Reflex motion reminiscent of:
def clear
ListingFilter.discover(ingredient.dataset.filter_id).destroy
@filter = ListingFilter.create
cable_ready.push_state(url: request.path)
finish
And right here you’re, as shut as ever to everlasting bliss in search paradise. You possibly can take a look on the reside app here.
Behold the afterlife
Let’s recap what we realized. Due to StimulusReflex , we realized the way to construct a brilliant reactive search and filter interface with clear and extendable code, nice efficiency, and nearly no JavaScript. We noticed how cable_ready might present some sprinkle of magic behaviour on prime of StimulusReflex. We have been capable of cleanly and quickly persist, then replace our search information due to all_futures. We additionally realized the way to chain conditional scopes in a protected method.
Sadly, good issues hardly ever final eternally. In our subsequent episode, we’ll see how new necessities and a much bigger set of data will occasion poop our not-so-eternal dream. You may get to see how ElasticSearch can save the day and permit us to construct the last word search engine.
Thanks for studying people, and see you on the opposite aspect!
