I’ve held off on this for a while, but starting to build it out.
Did phase one today, which was:
New DB table for photos, using ulid for non-sequential ids
TDD: Writing tests that endpoint returns a list of photos for a cafe, and excludes ones that haven’t been approved yet
Creating the model and factory, and adding a global scope to the model so that by default only visible photos are returned. Admin interfaces that will need to get all photos for approval can bypass that scope (which is verified by a test).
Next steps could be endpoints and UI to actually upload photos. It’ll be a while before anything is visible on site, because uploading itself has a number of components. I’ll need a form to upload, possibly with Ajax for better UX, a queued job to resize images and maybe create a WebP conversion too, copying to S3 and updating the DB.
Ideally there’d also be a cropping widget in the browser for the thumbnail view.
To begin with only I will be able to add images, but I’d like to open it up to trusted users too.
Will also need tools for deleting (including S3 objects, not just the DB records).
Seldom used, but important to have the option in the case the user has written something they don’t want public anymore.
I’ve written the first pass of the feature: Ajax request from the coffee history on the cafe page, server authorization with the Laravel policy and the simple CRUD delete action is working.
The other part of this feature though will be recalculating cafe data now that the input ratings have changed. I’ll see to reconfigure some of these jobs to recompute the cafe’s global ratings; the user’s own ratings at the cafe and the position on their most frequent list.
I do have an existing event listener that recalculates new ratings, but unfortunately it’s specific to adding new ratings. When a user deletes a coffee the input data will be different (one less rating); so I need to change this code to recalc from the remaining data set.
I’ve migrated the app to the latest version of Laravel. The Shift product helped a lot, but it still required a couple of hours of manual changes and testing.
I’ve progressed the search feature and would like the cafes listed in the results to use the components for the Favourites and To Try lists. The Vue components themselves can be dropped in, but I realised that they don’t have a default link to their underlying state.
Where they are used on other pages in the app I have loaded the data as part of the initial server response, which flows through to the components.
So now I have to either replicate that pattern on other pages, or refactor the components to use Vuex state and add an API endpoint to load in the data. The downside is it creates a bit more frontend work for me now, and also requires another HTTP request to load the full page data. The upside is more portable, self contained components.
In hindsight I can see why an SPA approach using Intertia would be helpful as it would remove some of this extra work, but I didn’t have the scope to build that way when I started this project.
I’m wondering if this refactor will also allow for fresher data on the My Cafes page. If user toggles a Fav/ToTry status on one route, it currently doesn’t automatically update if they switch Vue routes. If I move to central Vuex state that problem might be solved by default. So it’s a more robust solution.
I do like my colour scheme, but in bright outdoor lighting it can be too dark.
Imagine something like this:
20 second mockup in dev tools.
Implementation
I currently use Sass variables. Would need to refactor to CSS Custom Properties, so that I can refine them in scopes for a switchable mode. It would be fairly far-reaching because of that.
Also Vue and SVG components would need to be updated too. I might pencil this in for post-launch.
Oh my god, this feature has been like an iceberg that reaches the bottom of the Mariana Trench.
I thought I had a decent code outline before this Christmas week of development, but there was way more to it.
Location
I didn’t want string based matching of the location search term, because nearby suburbs, and relevant matches would be excluded. The feature uses a maps API to geocode the term into a lat/long position. It then uses the spherical distance formula to return a set of cafes within a reasonable distance. As the database grows I wanted to avoid huge DB table scans, having to calculate the distance for every row in order to get a result.
So to make that more performant I’m first calculating an approximate bounding box to limit the results using indices on the lat/lng columns.
The size of the area to search, and the number of results to expect also depends on the context. For example if you’re in Melbourne CBD you’d expect to get a good number of results in a close radius, and showing cafes 4km wouldn’t be useful. However if you’re in a regional Queensland town with much lower density of specialty coffee the radius might need to expand a lot more to show any useful results.
Searching with Cafe Name
This also complicates the search logic as it affects what might be reasonable search radius when used in combination with a location. The name searches uses a full text index against a separate table help fuzzy match terms (to avoid the user having to match it precisely).
This means some irrelevant cafes might be returned and I don’t want to limit the radius too closely or we might only return the wrong cafe, when the correct one was just a bit further away.
I also discovered the complexity of different location scopes. For example if a user searches for “Foster and Black” in “Brisbane” the geolocation lookup will centre Brisbane as being the centre of the CBD. Foster and Black might have locations in various Brisbane suburbs but they might not appear if those suburbs are too far from the CBD. The users intent would be to include them though.
I’ve attempted to solve this by expanding the radius when a name is included until either the max radius is reached, or we find cafes matching the name.
It’s difficult to know until I have more users and cafes if the search logic will hold up and be robust enough. Long term I might need to migrate to a dedicate search tool like Meilisearch or Elasticsearch if they also support geography.
Progress this Week
I’ve spent about 18 hours this week on this (along with a bit of infrastructure work around servers and build tools) and am pretty happy with the progress.
I have found inconsistencies in the data format being returned to the frontend though (depending on the search criteria). I’ll need to normalize that before building out the frontend Vue components.
I’m also wary of the UX if the user clicks away to a result cafe, then comes back to an empty search state. I might look at implementing InteriaJS so this can happen in an SPA style to preserve state. There could be some edge cases there in partially implementing it into a project that also uses traditional server page loads.
Miscellaneous Problems
This UI uses a location watcher for the browser’s geolocation API feature. I need to geocode that lat/lng too, and I found that in some situations the browser just keeps reporting either the same location, or a very subtly different location again and again.
To avoid thrashing my API endpoint that does the lookup (which itself caches/proxies the external API) I implemented a basic client side cache so that repeat requests for the same position are not sent to the server.
Testing
Unfortunately I need to refactor some of this work to be more testable. The geo lookup code a bit too embedded in other logic, so to test that would result in real external API calls.
I need to abstract it out (already used in two places in the app) and make it mockable.
Next Steps
I think the search logic is passable. I would like to incorporate more algorithmic sorting including the full text search ‘score’ of the cafe name, and other parameters such as rating and relevance to the user’s coffee history. That might have to wait for phase 2 though as it’ll add a lot of time to development otherwise.
The immediate step is to build out the result set frontend components so they look good and are usable. There’s some data massaging to go along with that so that individual cafe locations are combined into the parent cafe rows.
Beta Launch
And as far as I can remember the only other main features I need to launch are:
User ability to add a new cafe (and for me to review the data and approve)
Ability to edit a reviewed coffee (e.g. fix a mistake)
Would also really like more robust testing of the search feature.
I’m getting ready for a beta launch and have setup a production VPS. It’s oddly slower than the staging server despite being on the same (or very similar) plan from the same provider.
Both are 1 vCPU, 1GB RAM, NVMe SSD, Ubuntu 20.04 and PHP 7.4, but the older staging server is faster everywhere.
The new server was configured by Forge, instead of me doing it manually.
It can build and deploy the app in ~75 seconds instead of ~120. The queries run faster and it has lower memory usage.
I’ve also now benchmarked it with Sysbench, and again for both CPU and disk the staging server is faster.
Disk writing 8GB of files: Staging ~21% faster
CPU Prime numbers, 10K limit: Staging ~41% faster
CPU Prime numbers, 20K limit: Staging ~38% faster
App build and deployment: Staging 38% faster
Update: Having tested on a 2 vCPU server, my old Ubuntu 16 box, and a Vultr High CPU plan it seems the slower prod server is closer to normal, and my old staging one is an anomaly.
