Ok I’ve cracked it! The Scout driver is more limited in the query construction it can support.
I’m breaking out to use the Meili client directly and can build up my queries with a lot more flexibility.
Migrating Search to Meilisearch or Similar
I began a partial migration to Algolia for cafe search a while ago. That was a spike that only covered some of the simpler uses.
As part of the work to make location search easier with suburb autocompletion I’d also like to move the suburb database into a search index. The free tier of Algolia is limited and it’d blow away my quota, so I’m looking at Meilisearch (MS). That’d mean the cafe search would move into it as well.
Initial testing was ok, but it seems that MS does not support multiple geo location positions per document in the same way that ElasticSearch does. This would have been extremely useful, as my data model has a one to many relationship between cafes and their individual locations. The search would be easier to build if MS could internally handle that.
If I’m right, I’ll either need to do a hybrid, multi-step search to find cafes within a location, and then another with those ids to handle the main search which might sort by score and other ranking factors.
Going full ElasticSearch is also not that appealing because it’s not supported by Scout, so I’d have to do more custom data mapping, querying and syncing.
Much to consider.
Allowing Users to Add Cafes
This was always part of the plan, however there are a few steps required to get this going. As an admin I have already coded most of the functionality, however for non-admin users I need to consider:
- Suggesting existing cafes they may have missed
- Being able to merge cafes for when duplicates are created
- Verifying the user entry to fix mistakes, add additional optional info such as social media profiles for the cafes, or even cleaning up or monitoring spam. Creating a cafe essentially gives some entity a web page on my app, so it could be a target for abuse.
- Adding more locations to an existing cafe
- Suggesting that cafes or specific locations have closed down (happens quite a lot, especially with covid).
Right now my Controllers are fragmented with a frontend and admin equivalent. I need to move the ::store method to the public one, use a Form Request to validate and handle the slight differences in behaviour depending on who is adding.
Also need to ensure the geocoding job is queued in either case, and possibly monitor usage as there could be an API cost if it’s abused.
MVP
To get something going soon I think I’ll let users add cafes until there is abuse. I might throttle them to adding N in some number of hours/days.
I’ll review additions manually, and later when I can I’ll build the merge tool.
Still so much work for what seems a simple CRUD feature.
Rating by Type of Coffee
Not all cafes will make white, espresso or filter coffees equally, or equally well. Some might have a focus on espresso and bit a little average for the whites, or vice versa. Rather than combining all their ratings and losing that granularity I will let users specify what sort of drink they had when rating a coffee.
I think this will be important to help give better recommendations. A cafe might do stellar cappuccinos, but if they’re ordinary at a Long Black and that’s what you’re going to order the Cap won’t help you!
The feature does add quite a bit of complexity to the development though. I need to start tracking ratings separately for the drink categories, as well as in aggregate. The scoring algorithm will also ideally weight itself towards a users history and preferences too.
It also has frontend implications with extra UI, state management and icon design requirements. The image above is still, but I am working on building subtly animated SVG icons for the drink types to enhance the UX of choosing them. I’ve got a nice little stepped animation to show the selection action.
SVG
I love the flexibility of images-as-code and being able to add simple animation effects with CSS. I am a little concerned that bundling the SVG icons into Vue components will be adding more weight to the final JS build.
I think referencing a static SVG as an image will prevent me from being able to style and animate individual shapes with CSS. It needs to be an embedded object. When I’m already deep in a Vue component structure that means the icon also needs to be a Vue component (as opposed to server side injected SVG code).
In my defence though, I think most of the audience who are into visiting specialty coffee cafes for entertainment probably err towards having high end phones that are capable of dealing with the load. The JS will be cacheable and probably even service worker cached in the future, so it’s more a CPU/parsing issue.
Backend
Most of the backend adaptations are still to be done though, and the actual search weighting will happen later as I build that feature.
Glad to be Moving to Vuex State Management
The growing amount of data stored for a rated coffee would get difficult to manage within a single parent component. I’m glad I’ve started the work of using Vuex to control the global state. Passing this many props and events up and down the component chain would be quite messy.
Overall Progress
So much of the project framework is complete, but there are a few key features left to build.
- Drink Type management described here
- Reverse geocoding and caching so the app can refer to your suburb/city and not just Lat/Long
- The main cafe search feature, using that geographic information and drink preferences
- Ability for users to add missing cafes at the time they are entering a review. (This can use some of the existing backend code I have already written from an admin perspective)
- Service Worker / PWA bundling. This will be very minimal for now. Home screen, and basic asset caching, but no real offline support.
- Performance and A11Y review
Images
Coded a basic cafe banner image handler today.
- Resizes to 2000px and 1000px variations
- Creates WebP version for supporting browsers
- Had to use GD for that, my install of Imagick didn’t have WebP support
- File upload is sync, but resizing and cloud upload is Queued job so user doesn’t wait
- Uploads to S3 and cleans up old file versions
The code that renders the image on the frontend cafe page is suuuper rough. Just gets a random size, rather than using srcset. Sniffs webp support and does cloud disk access right in the view. Definitely needs tidying up, still a good start though.
Sourcing 2000px images (wide on retina) might be difficult. A lot of photos on the web seem to be lower quality. Might have to ask users or cafes to submit a quality one!
Dev Stats
JS not included. What’s a good package to count that?
Ratings, Favourites, A11Y, SVG & Lists
I made a nice chunk of progress on the ‘To Try’ feature of HadCoffee this weekend. This lets you bookmark cafés you’ve heard of and would like to try later. By default the list on HadCoffee will sort by proximity using your current location.
Most of the recent work for this feature was on some nice-to-use buttons for toggling a cafe in your Favourites or To Try list. I had mostly built the ‘Fav star’ earlier, but improved it with a visual :focus indicator for keyboard accessibility.
See the Pen SVG Favourite by Mike (@mike_hasarms) on CodePen.
The coffee cup icons are SVG based Vue components that receive rating data from the page and fill the cup appropriately.
I also worked on some basic animation of the button to add a cafe to the To Try list to make the transition to the active state a bit more obvious. I think little touches of movement make the thing more engaging to use as well.
Accessibility
I spent a bit of time improving the A11Y of these (and other features). The Vue SVG components are keyboard accessible, labelled and indicate focus. I am planning to do a more thorough A11Y test before I release, but I’m certainly not leaving all of those concerns until the end of development.
Anonymous Users
Currently the Favourites and To Try lists require a user to be logged in. I had planned to allow anonymous users to begin keeping these lists for themselves in localStorage. The idea was to lower the barrier to entry to using the site. It does however add development complexity as the app needs to be able to use both local and server storage for those lists and load in data in different ways.
As this is a mostly server-rendered app all the initial page data is sent down with the page load, and populated into the Vue components. Supporting anon users will require separate logic to request their cafe data from an API endpoint.
That’s not hugely difficult, it’s just another thing to be done before launch.
Cafe Suggestions with Geo Search
I’m keen to make some solid progress on HadCoffee over the Christmas period to hopefully get to an early 2019 v1 launch. This week I was working on the main coffee rating interface which depends on users being able to select or enter the café they visited.
I’ll add the autocomplete lookups for typed cafe names, but I also wanted some basic quick suggestions based on where the user has been before and their current location.
This would save typing for common places and maybe also hint at other nearby cafés the user mightn’t know about.
The frontend part of these feature went pretty smoothly. The Vue component watches for the users location (HTML5 Geolocation) and calls the backend to load these quick suggestions as it’s known. There’s a little debounce action to slow things down as the geolocation API can return updated positions in quick succession as it locks on a more accurate location.
Geographic Search by Proximity
The smooth progress I was making through this feature hit a wall when it came to actually querying cafés by distance though. I was hoping to use MySQL’s ST_Distance_sphere function to let the DB do that work. I’m running MariaDB though, which although it’s advertised as a ‘drop in’ replacement for MySQL does not support this feature ????
I prefer a simpler dev environment (I’m not using Laravel Valet or Docker images) so I didn’t feel like swapping to MySQL for this project. Changing my workflow to use Valet also wasn’t very appealing when I’m otherwise happy with the setup. so I briefly tried migrating to Postgres. I know it’s a great DB, but I haven’t used it before and that’s a big change to have to make to run one type of query.
In the end I’m going with a raw SQL query to help with this. I’ll add a simple bounding box to its parameters first to avoid having to do a table scan of every cafe in the world (once my DB gets to that point ????)
Although it took a windy path this geo search will also provide the basis for the other cafe search features on the site such as the autocomplete (to improve relevance) and the location based search.
$query = "SELECT id, cafe_id, lat, lng, address, locality, city,
( 6371 * acos( cos( radians(:lat) ) * cos( radians( lat ) )
* cos( radians( lng ) - radians(:lng) ) + sin( radians(:lat2) ) * sin(radians(lat)) ) ) AS distance
FROM cafe_locations ";This is a good step towards being able to add café & coffee reviews, however the next big sticking point will be letting users add new cafés as they go.
Ideally I’d like to collect a bit of meta data such as roasters, menu and seating options to help users finding cafés, but I’ll have to see how much data entry users will tolerate. I also need to be aware of how or if I can verify this community sourced data.
Development
Dev is underway. Frontend template is largely built, but will create components as needed, rather than building a complete styleguide and system up front.
Backend dev has begun too; the Laravel backend started, user auth, and initial work on the process of adding cafes. That’s got a few extra steps than a basic CRUD form because I want to geolocate cafes without the user (or me) having to manually lookup their location.
I’m using HTML5 GeoLocation + Google APIs to reverse geocode the user’s current city for a hint as to which cafe they mean when entering data; then another reverse geocode to get lat/lng and address for the cafe in question.
This will let me do some initial data entry without as much tedious Googling and copy/pasting.
It does mean I need to get a bit diverted on the structuring of the application JS though. I don’t want to go to far building ad-hoc hack scripts that are hard to organize later. I need to workout how my application JS will combine with Vue, Vue Components and future code. May also need to consider how I cache user location or reverse geocode results to reduce API demands.