Announcements/UserFrosting

UserFrosting 4.2 🥳

This has been a long time coming (no thanks to me, more on that later).

It represents 1 and a half years’ worth of work, and the first stable build!

So how did we get here?

Unit Testing 🧪

@lcharette championed this so I don't have too much to say other than that this, in my eyes at least, is the single most significant addition in terms of giving the confidence needed to ditch the alpha tag.

Through unit testing numerous bugs were discovered and squashed (particularly around handling of file paths when development was wrapping up), and in a few instances introduced regressions were caught and fixed.

Capacity to test sprinkles has also been improved.

Assets Pipeline 🔧

The build pipeline in 4.0 and 4.1 was essentially hacked together, and when things didn't play out right you could tell.

  • Difficult to debug issues
  • Seemingly minor updates breaking everything
  • High IO usage
  • Intermittent failures
  • Dependencies unable to be updated
  • Unit tests completely broken

Those are just some of the problems faced in the older versions, all in an area I'm responsible for. To put it mildly, I'm not proud of what was shipped before. Turning this around required a lot of effort and for a time left me burnt out.

The journey begins

Before landing on the backwards compatible rewrite we have now my first idea was to create a new highly extensible stream-based bundler that would offer adapters to convert other configurations, such as for gulp-bundle-assets. After a few months of intermittent planning, and false starts it was ultimately deemed unfeasible. The concepts drafted up offered no clear path for comprehensive test coverage, at least for my experience level at that point in time.

Taking what I had learnt from my first foray, my second attempt focused on simplifying the system as much as possible. To achieve this, the gulp-bundle-assets fork was up for a complete rewrite, top to bottom. Unfortunately, around this time uni studies ramped up and wore me down. Twas not a fun time.

Mixed throughout the bundler fuss, support for packages from NPM (using Yarn), deduplication of Bower dependencies, and a massive overhaul of the UserFrosting\Assets package was introduced. A great addition which greatly reduced build times but was not yet implemented in a polished form. The dependency deduplication work spawned @userfrosting/merge-package-dependencies.

Work on UserFrosting\Assets bears a special mention. While the majority of the UserFrosting project follows Romantic Versioning, the scope of the overhaul represented enough of a shift to trigger bump to the generation number and more notably the chance to move the library onto Semantic Versioning. There are 2 3 primary reasons why the leap was made;

  1. The rebuild was designed with forward compatibility with future UF versions in mind, primarily through extensibility.
  2. The intent is that the library can be consumed by projects other than UserFrosting to open potential for a snowball effect and demonstrate a reason for it not existing in UF core (besides transparent updates).
  3. I'm super stubborn!

Regarding point 2, one of the more significant changes for realising this value was to integrate the development asset server. Jump ahead to now however, and an unexpected perk is that it has offered a significantly easier way to handle feature upgrades in that part of the system. We are now in a position where we can implement major improvements without necessarily requiring a manual UserFrosting upgrade.

Seeing things through

Come late 2018, I was rediscovered that which I had longed for during my major uni project. MY FREE TIME!

Better yet though, over the course of the year I'd had the chance to hone my skills and get some perspective. So, of course naturally I went to work milling away at my list of projects, sans UserFrosting. Or at least until the dev chat pinged me to make sure I hadn't kicked the bucket since last I checked in. What can I say? I'm very easily distracted! 🤷‍♂️

So, returning to UserFrosting the first order of business (aside from figuring out the state I had left everything in) is to get @userfrosting/gulp-bundle-assets sorted. While the specifics around how I got it sorted are now lost to time, the technologies used are crystal clear (cause I’ve come to love).

  • TypeScript - 😻 Static types!
  • ava - By far, the cleanest and most predictable unit testing framework I've ever used!
  • esm - Debugging published source that is heavily compiled is a drag, so this was a godsend!

Long commutes

Something unexpected happened later in 2018, my casual search for a part time job ended up resulting in a full time position. Not a leap I was initially open to but I'm glad I caved. Beyond the huge amount I've learnt since starting at 4mation, the one-way 2-hour commute provided me with a solid block of time focus on finishing up my part of UserFrosting 4.2.

Significant progress around stability during this this time thanks to;

  • End to end unit testing which flushed out innumerable bugs within the bundler rewrite.
  • Addition of test coverage checks which helped to flush out edge cases and dead code.
  • Comprehensive internal documentation which helped to audit the functional correctness and responsibilities of various components.

Throw in the last second addition of @userfrosting/browserify-dependencies to open compatibility with most of the Node ecosystem plus the usual alpha → beta → rc → stable release process for all the worked-on components and you've got a robust asset pipeline that was sorely needed!

The Little Things ⚛

  • Site visitors will be shown the best possible locale match based on browser preferences. This extends to the locale dropdown during registration in the Account sprinkle.
  • Greek locale
  • More bakery commands, command improvements, and API refinements.
  • More database query helpers.
  • Vagrant integration.
  • Improved Docker integration.
  • dev -> debug.
  • More mailer options.
  • Refactors, optimisations, QOL enhancements, and too many bug fixes.

Retrospective 🤔

No shortage of lessons to be learnt from this major release cycle.

🌳 Branching Strategy Adherence

Branching within the UserFrosting project has always been a struggle. The temptation of committing straight into the develop branch so that others might get to see and test your work sooner almost impossible to resist! But resist we must, and we did not, so we paid the price.

  • Inability to delay components.
  • Disruption to others.
  • Frequent merge conflicts.

These are just some of the problems faced.

Part of this plays into a dependence on others to progress, because before you merge a branch in you want to know that you’re not breaking someone’s workflow or changes. However, an open source project like ours doesn't have anyone around to work full time, meaning getting changes in can at times take a long time. If you've got the power to merge, it becomes difficult to resist.

That said, the problems caused by partaking in the sine of merging your own work in and committing directly have, for me at least, killed any of that temptation. For the time being I'm back to the self-enforced restriction, however independently I'm testing GitHub's branch permissions in gulp-uf-bundle-assets to see if adherence can be mandated by the system in a manner that won't drive everyone crazy in the process.

🏃‍♀️ Momentum in Open Source

Momentum (by which I mean the rate at which anything workable goes from idea, to concept, implementation, and finally live) is like the heartbeat that keeps projects like UserFrosting alive, retaining interest and gaining confidence the longer it is kept at a reasonably high level. To establish and persist momentum planning is essential, plus a few extra points that fold into making an effective and achievable plan.

  • Define an MVP consisting of deliverables that have a high probability of being delivered.
  • Specify a loose deadline that is not too far into the future, as a rule of thumb within 3 months subject to the scope.
  • Add any nice-to-haves/stretch goals that could be achieved within the timeframe but are not essential, nor will block the release.

Before 4.2 there was momentum, unfortunately a lot of that was lost due to the assets work becoming a critical component that was added into the main working branch (as mentioned above), leaving everything in a bad state while I was unavailable. This leads into a 4th and final point.

  • The MVP must remain achievable if someone becomes unavailable.

So how do features that require the work of someone who might disappear then? Track them as a stretch goal if they might be achievable, otherwise slide them into an MVP when they are complete. Better yet, release it when its ready and bump of the projected number of the next milestone. The important part is that value is being continually delivered.

🐘 Scope

One of the great things about community contribution driven open source is you can spend hundreds of hours getting something just right. It can however also be the worst. When the sky is the limit, it can be very easy to go overboard on the scope. Aside from the obvious extended development cycle, consequences include;

  • Burn out especially in complex systems and when value is not realised over an extended period.
  • Wasted effort if an alternative presents itself.
  • Unsustainable systems that are too complex for anyone else to contribute to, defeating the purpose of open sourcing.

The scope thankfully did collapse down into something more manageable in time (and something sustainable), however there were still components that could have been delivered later without sacrificing capacity for future enhancements. Bumpy ride, but live and learn.

Looking Ahead

There are a number of improvements that are on the horizon, many of which build on the work done in 4.2.

That said a lot of the changes within UserFrosting itself will be stuck in limbo until the unit testing situation is improved. As a priority, we don't want to introduce regressions now that we've finally shaken free of alpha.

Jordan Mele

Jordan Mele

Site owner

Read More