Oh my god, this feature has been like an iceberg that reaches the bottom of the Mariana Trench.
I thought I had a decent code outline before this Christmas week of development, but there was way more to it.
Location
I didn’t want string based matching of the location search term, because nearby suburbs, and relevant matches would be excluded. The feature uses a maps API to geocode the term into a lat/long position. It then uses the spherical distance formula to return a set of cafes within a reasonable distance. As the database grows I wanted to avoid huge DB table scans, having to calculate the distance for every row in order to get a result.
So to make that more performant I’m first calculating an approximate bounding box to limit the results using indices on the lat/lng columns.
The size of the area to search, and the number of results to expect also depends on the context. For example if you’re in Melbourne CBD you’d expect to get a good number of results in a close radius, and showing cafes 4km wouldn’t be useful. However if you’re in a regional Queensland town with much lower density of specialty coffee the radius might need to expand a lot more to show any useful results.
Searching with Cafe Name
This also complicates the search logic as it affects what might be reasonable search radius when used in combination with a location. The name searches uses a full text index against a separate table help fuzzy match terms (to avoid the user having to match it precisely).
This means some irrelevant cafes might be returned and I don’t want to limit the radius too closely or we might only return the wrong cafe, when the correct one was just a bit further away.
I also discovered the complexity of different location scopes. For example if a user searches for “Foster and Black” in “Brisbane” the geolocation lookup will centre Brisbane as being the centre of the CBD. Foster and Black might have locations in various Brisbane suburbs but they might not appear if those suburbs are too far from the CBD. The users intent would be to include them though.
I’ve attempted to solve this by expanding the radius when a name is included until either the max radius is reached, or we find cafes matching the name.
It’s difficult to know until I have more users and cafes if the search logic will hold up and be robust enough. Long term I might need to migrate to a dedicate search tool like Meilisearch or Elasticsearch if they also support geography.
Progress this Week
I’ve spent about 18 hours this week on this (along with a bit of infrastructure work around servers and build tools) and am pretty happy with the progress.
I have found inconsistencies in the data format being returned to the frontend though (depending on the search criteria). I’ll need to normalize that before building out the frontend Vue components.
I’m also wary of the UX if the user clicks away to a result cafe, then comes back to an empty search state. I might look at implementing InteriaJS so this can happen in an SPA style to preserve state. There could be some edge cases there in partially implementing it into a project that also uses traditional server page loads.
Miscellaneous Problems
This UI uses a location watcher for the browser’s geolocation API feature. I need to geocode that lat/lng too, and I found that in some situations the browser just keeps reporting either the same location, or a very subtly different location again and again.
To avoid thrashing my API endpoint that does the lookup (which itself caches/proxies the external API) I implemented a basic client side cache so that repeat requests for the same position are not sent to the server.
Testing
Unfortunately I need to refactor some of this work to be more testable. The geo lookup code a bit too embedded in other logic, so to test that would result in real external API calls.
I need to abstract it out (already used in two places in the app) and make it mockable.
Next Steps
I think the search logic is passable. I would like to incorporate more algorithmic sorting including the full text search ‘score’ of the cafe name, and other parameters such as rating and relevance to the user’s coffee history. That might have to wait for phase 2 though as it’ll add a lot of time to development otherwise.
The immediate step is to build out the result set frontend components so they look good and are usable. There’s some data massaging to go along with that so that individual cafe locations are combined into the parent cafe rows.
Beta Launch
And as far as I can remember the only other main features I need to launch are:
- User ability to add a new cafe (and for me to review the data and approve)
- Ability to edit a reviewed coffee (e.g. fix a mistake)
Would also really like more robust testing of the search feature.