Pilcrow's Semantic Search
Pilcrow is an AI-sprinkled note taking app, like Apple Notes on steroids.
It's aim is to help you create a 'digital garden' - an evergreen notebook where you plant a seed of an idea and watch it grow. Existing note apps are more based around project management. They focus more on organising and storing your thoughts for specific use cases rather than open-ended learning. Pilcrow encourages you to write one idea per note. This is because the very act of writing a note bubbles up new ideas and helps you discover whilst you write. Notes are seperated into lots of seperate ideas which you can connect together.
Creating hundreds, perhaps thousands of notes in most apps can be a nightmare to organise and taxonomise. Endless filing and organising takes lots of precious time away from what you're in this application to do: write.
The idea
The beauty of text-based documents is that they can be searched very easily.
However, search patterns are often poorly built and can be fustrating as they can form a barrier between what the searcher actually wants and what the search can turn up with keywords (more on this down below).
Foundations
It's important that our search adheres to a few principles so that it not only functions properly but provides a useful and pleasurable experience. We're putting most of our eggs in this basket as the main way to find notes in your digital garden.
- Available: Search UI should show that it's ready whenever you need it. Users shouldn't need to "search" for the search bar. It's frequently the key entry point to starting a new task or even an escape hatch if the user get's lost.
- Helpful: Obvious but important. Search UI can be daunting because there are almost infinite things you could search. We should help users understand the parameters of what they can search for.
- Fast: Something I've tried to bake right into Pilcrow across the board – search being slow is awful. Expectations of fast, responsive results are common place and create a poor experience if not met.
The entry point to search is a text input with a icon to signify it's use case. Sitting at the top of the UI visually indicates that the search covers everything below, in this case – all your Pilcrow notes.
In reality, that input is just a big button. When clicked it opens up a dialog view that sits directly above that initial 'fake input'. This creates the illusion that the dialog UI has surrounded and expanded out of the input. Magic!
Keyboard shortcuts play an important role at increased usability. Hitting / and Esc opens and closes the dialog respectively, whilst also automatically focusing on the input on open.
Starting to search on change (as the user types) rather than clicking a button labelled 'Search' or something similar is a common pattern that often makes the search feel more immediate.
However, you can cause yourself problems as with each character change, you send requests to your backend. This can overload things, making things slow and cumbersome.
I added a debounce to the Pilcrow search, waiting 750ms until the request is fired. This gives the user enough time to stop typing but not enough that it feels unresponsive.
There's nothing special about this value, it's just what felt right.
Under the hood
I'm not a backend guy.
Pilcrow uses Supabase, which in turn is built on top of Postgres.
Adding a text-based search uses tsvector, which is built into Postgres.
I added a column called plain_text which would remove all the node decorations and things from the article and just display the text.
Whatever the query was in checked against this and the title.