BP Core Frontend Templates Refactoring

[hnla]

This ticket proposes an initiative to begin the process of refactoring the core template files to a new level of standards that clears up some of our legacy issues with markup, core markup, classes and general structure.

It would encompass four main areas.

• Template files markup and classes(tokens): Each file stripped back and re-built focusing on a leaner structure, improved tokenism and accessibility improvements.

• Core files frontend code: Identifying & extracting - where possible - all markup from core to template files, also in addition, to sweep through all template tags identifying those that ought to be able to pass arguments with a view to enabling bp_parse_args for them and any obviously sutable paramaters.

• Creating template parts: In tandem to the core markup issue might be extracting certain markup elements such as filters and or search to thier own get_template parts allowing us to manage these elements easily from the frontend templates.

• Stripping back assets: Essentially taking the opportunity to deep clean the JS files, removing all but essential JS ?Cookie functionality and perhaps re-factoring JS into separate files by usage type.

Frontend Template files (legacy to new)

As this needs to be a unfettered refactoring the new templates would effectively cause the apple cart to overturn where existing themes had overloaded templates thus I propose a mechanism whereby we effectively default new installs to using the new templates, existing installs and possibly overloaded templates would be automagically checked and if found would be switched to using the Legacy templates. We have pretty much all the parts needed already I think to be able to setup a checking process so don't think this aspect would prove as hard as it may sound, further a selection option would be provided that allowed users/devs to select what templates used - here we would have some logic puzzles to deal with, existing site installs would be switched back to Legacy templates, however they may choose to update and switch to the new in which case then we need to capture that decision as a check before we do any switching of existing installs.

This approach to templates could afford us the opportunity to follow a similar paradigm to the WP default themes whereby we could roll out new templates and corresponding styles on a yearly or irregular basis but be able to do so safe that it then becomes a user choice to use them and those new templates are free of any concerns in preserving legacy behaviours.

Core Markup

I envisage a master ticket that attempts to identify and list these core frontend markup objects as found; one might be generically 'Dir Search Forms' another might be 'Messages/Notifications Bulk Select Markup' we'll locate it in the core files and once tacit agreement is arrived at to proceed on one of these it will be branched off to it's action ticket to flesh out the details and agree a method for re-working to frontend template parts where applicable.

I think there has been a generally acknowledged acceptance that core holds too much frontend template markup, bloating core files, making it harder to do simple re-factoring without having to break off, create a function, return as a filter etc.

In some cases we actually have duplicate markup effectively; bulk message/notifications selects is a case in point where markup exists in core bp-*-template.php for respective components but where in reality the markup is the same, the only difference being the select control 'name', so it seems to make sense we could extract this to one include file and use current_action to provide correct control 'name'.

Core will benefit slightly from lighter files, less code to wade through, developers will gain easier access to markup to modify if they wish.

I don't say some of these won't require a little more work than simply copying out, some will have possibly a series of functions/filters that might need re-thinking.

I think this task will greatly help future template implementations and provide what we'll all probably agree as better separation of core functionality from template functionality and ease some of the frustrations in working on frontend templates.

It's possible the task may span more than one release cycle (there's a little more work than it might at first appear), but once we have a clear master list of tasks agreed on we then have the opportunity to group them into sets for a release if it does look like spanning release cycles.

[DJPaul]

We've spoken about this before, so you know my views. I'd be interested in getting some examples of current bad template code where we could make some of the improvements you've mentioned.

[hnla]

Would rather we didn't refer to template code as 'bad' though :) not a phrase I've used here and to some extent overlooks the very many template patches and areas identified over the years that I've added or highlighted, our templates aren't 'bad' they just need a refresh and refinement. Some of the detail on what needs looking into is in the above outline, as for specific examples I'll see if I can add one or two when I have a further moment, however a few examples should not be the basis for judgement overall, there is a larger picture to view on which we must base judgements.

[mercime]

This is quite an interesting and bold proposition. I'm all for improving accessibility everywhere (#6531 & #6532) as well as many of the points you mentioned above. But isn't it better to start the ball rolling and easier to review and test via a new Template Pack plugin like the ​Template Pack project started by karmatosed and yourself some time back?

[hnla]

But isn't it better to start the ball rolling and easier to review and test via a new Template Pack plugin

Yes very possibly. That initiative though never resolved the question of how new files would be implemented, at the time I was concerned that we should have the process for implementation resolved or at least a notion of how it would be integrated also we approached things with a major focus on display, with this process I'm more interested in separation of presentation from content in other words addressing the templates with a view to a robust and stable set of files that can then be styled; initially with a thinned down core stylesheet although equally if people are willing with more detailed styles for BP elements.

[boonebgorges]

hnla, thanks for starting this conversation. Your specific bullet points - moving markup out of core functions, breaking templates into template parts, etc - all seem worthwhile to pursue.

This approach to templates could afford us the opportunity to follow a similar paradigm to the WP default themes whereby we could roll out new templates and corresponding styles on a yearly or irregular basis but be able to do so safe that it then becomes a user choice to use them and those new templates are free of any concerns in preserving legacy behaviours.

I'm skeptical that this is viable. First, WP's team of contributors is larger than ours by orders of magnitude. Not only that, but the yearly WP themes - at least in their initial design and implementation - are pretty much commissioned by Matt. I don't feel confident that we can muster the wherewithal to keep up this pace on the BP team.

Moreover - and this, to me, is the crux of the template problem - frequent breaks with backward compatibility (in the form of new default templates) result in multiplied maintenance tasks for the core team. Think about bp-default and bp-legacy: every time we add a new feature to bp-legacy, we get complains about how it doesn't work in bp-default, and for a good year and a half we actually added the new feature to bp-default. How would we manage this with three or four or five default template packs? We'd have to have an EOL policy for our own sanity, but users would be frustrated by such a policy, as they don't want the onus of having to change their templates every year or two.

For these reasons, I support the idea of writing new templates, but I want to make sure that when we do it, we make it count. The basic layout of most of our templates has been the same since bp-default was introduced in BP 1.1 or 1.2. That was five years ago. What have we learned about our component UIs since then that could benefit from a sweeping change? (Think about the multi-pane Messages interface that was discussed in the turtleshell project.) What changes have taken place on the web that might affect the way we think about templates? (Do we, say, build in native support for header images, like Facebook and Twitter have recently done?) Etc.

with this process I'm more interested in separation of presentation from content in other words addressing the templates with a view to a robust and stable set of files that can then be styled; initially with a thinned down core stylesheet although equally if people are willing with more detailed styles for BP elements.

I realize that this is your response, but my point is that once you force people to a new template pack, it doesn't matter whether your changes are related to UI, or to underlying tech, or both, or neither: the damage is done. So let's only do the damage once, if possible, and solve as many of the extant problems as we reasonably can.

I should note that I'm not necessarily calling for a bottom-up redesign of BP UI. If that's too tall an order - which the sputtering nature of the Template Pack project suggests - then let's by all means do something more modest. At the same time, let's do it with a recognition that a major break in back compat is something we want to do only very rarely.

Sort of a side note: Some of what hnla suggests here - like more focused template parts - will mean that, in the future, we can more easily improve/modify/refactor the templates *without* compatibility breaks.

[hnla]

@boone thanks, yes I actually agree with your comments in the main and we needed this counter to my musings to balance things.

In respect of the notion of running in a sense a WP style template approach:

I'm skeptical that this is viable

I agree entirely on those points.

Moreover - and this, to me, is the crux of the template problem - frequent breaks with backward compatibility (in the form of new default templates) result in multiplied maintenance tasks for the core team.

Yes and that would be an entirely retrogressive step helping no one and is something we must avoid.

As for forcing users into new templates we would always have this issue with anything newly introduced or changed of breaking what might be established, it's that which is why, really, I mentioned the approach of defaulting new installs to new templates but checking existing installs/upgrades and switching them back to the original legacy files, it does mean keeping two sets of templates but not sure there's better way of moving things forward?

The initial proposal should not be considered modest though, in fact the reverse is true it's a very deep sweep through templates, the focus on separation of content from presentation is intended to highlight the essential principles of CSS/HTML where the document modal for a given rendering of data has not many choices in it's structure and constructs but just one ideal description of the content, styles are then meant to be able to re-define that visually in whatever fashion they see fit.

Thus what we attempt to do is setup up a series of templates that deal with the content in the best manner possible, we extract core markup and re-locate to new template parts, we attend to the perfect round of classes/IDs ensuring we have correct classes for generic elements then ones for specific elements (currently our navs miss that ideal ).

Having achieved that to a large degree that never needs to be re-visited ( added to maybe with new components or templates but not having necessarily to go back to messages because something isn't quite right.

Styling or UI/UX is really a secondary phase, one I would love to see tackled but one that is not mutually inclusive to the first process. In the first instance I would envisage a sweep through the existing stylesheet adjusting / updating rulesets for classes and IDs further thinning out our styles to carry a little less specificity( a problem on occasion with the companion styles) We are then setup to either leave a modest amount of styling or to look to tackle things to a greater degree and seeing just what we can do to spice up the UX of aspects like the messaging component.

So I think we should forget the notion of:

This approach to templates could afford us the opportunity to follow a similar paradigm to the WP default themes whereby we could roll out new templates and corresponding styles on a yearly or irregular basis

It was a passing thought only but with too many issues within it.

[boonebgorges]

Styling or UI/UX is really a secondary phase, one I would love to see tackled but one that is not mutually inclusive to the first process

But this is what I want to avoid, if possible. Imagine the following:

1. We currently support bp-legacy.
2. In BP 2.5, we introduce the bp-refactored template pack, along the lines of what you're suggesting in this ticket.
3. In BP 2.7, we introduce the bp-totally-new template pack, with revamped UX.

A couple of things follow from this:

• By BP 2.7, the BP team is supporting at least three sets of template packs.
• Anyone who upgrades from bp-legacy to bp-refactored, and customizes their site based on these templates, will be unable to upgrade to bp-totally-new without redoing all their customizations. Many tears ensue.
• Anyone who installs BP 2.5 or 2.6 using bp-refactored and customizes those templates will be unable to upgrade to bp-totally-new.

If it's possible for us to avoid this scenario and still accomplish our admirable goals of refactoring and redesign, I would strongly prefer to do so.

I know that it'd make your task easier to have carte blanche in refactoring, with zero concern for backward compatibility. But it may be worth considering which of your desiderata can be achieved without compatibility breaks, and which cannot:

• Moving markup generation from functions into templates (a job that is going to be extremely unpleasant, by the way) probably *can* be done without breaking backward compatibility.
• Breaking up our templates into smaller parts probably *can* mostly be done without breaking backward compatibility.
• Changing selectors and the structure of markup mostly *cannot* be done without breaking compatibility.
• Improving JS is a huge job that touches many things, but much of it (such as the cookie refactoring) definitely *can* be done without breaking compatibility.

IMHO, the guiding principle should be: If it can be done in bp-legacy, it should be done in bp-legacy. If it cannot be done in bp-legacy, it should be done as part of a full rebuild that includes a redesign.

[hnla]

The support question does appear to be the crux of things and as always that or backpat, whatever term we wish to append to this notion of maintaining working installations is the one where we shy away from hard decisions?

I get all the problems addressed above, but not sure what the solution is that keeps everyone 100% happy or if that is actually feasible.

We currently support bp-legacy.

We do, but why! And for how long do we? Are we simply going to indefinitely support versions? what is our cut of point for support, lets actually determine when we unburden ourselves of this?

On this question of versions, support and customised sites/ overloaded templates as I've mentioned before I think, to some extent we ought to view these circumstances of customised templates as carrying with them an implicit acceptance to maintain those custom templates in the face of whatever changes or updates BP pushes through; that leaves the real issue as one where an install today using unmodded legacy files upon updating BP finds the element rendering has gone and changed and they weep - this point is taken.

Styling or UI/UX is really a secondary phase

I only really labour this point to try and ensure that the notion of content separation is understood and that what this proposal is really about is not a fun one about style, layout and cool things ( lord knows I'd give my eye teeth to get back to doing interesting CSS in general in working life) but about the scaffolding we use to hang those styles on, if we didn't actually tackle any particular styling we would be having to track through the default styles updating any rulesets necessary to ensure that in fact the transition didn't upset things. Labouring things even further:

Anyone who installs BP 2.5 or 2.6 using bp-refactored and customizes those templates will be unable to upgrade to bp-totally-new.

1. The proposal was always really about a set of templates that didn't change - bp-totally-new would suggest styles, presentation, as the templates have achieved 'perfection' :)
2. If BP adds new components, creates new screens or new template tags etc, then this is not a new issue this issue has always existed; anyone who customizes or overloads templates faces this issue of having to maintain and update ( I know I've had to often enough) so nothings really changed?

I know that it'd make your task easier to have carte blanche in refactoring, with zero concern for backward

I think there comes a point logically where always having to keep one eye on backpat is simply too frustrating, whether that be core mid tier or frontend, we have concerns in both areas in BP. I think what I'm trying to say with this is if one wants to do the right thing, to do a job properly then there comes a point where one needs to work without one hand tied behind ones back. certainly in nearly every patch I've submitted over these years there's a frustrating burden in having to examine a change from so many perspectives, often when a change is straightforward and justifies no more than a ms of ones time. :)

Which desiderata can be achieved without upsetting backpat?

• Moving markup generation from functions into templates (a job that is going to be extremely unpleasant, by the way) probably *can* be done without breaking backward compatibility.

I have looked at many elements of this, and with some have already stepped through the exercise with Status and the template pack and other off project exercises and yes this set of tasks is one that doesn't have backpat concerns, it's moving the location and possibly adding classes or ability to pass args back but not one of changing the resultant rendered markup.

a job that is going to be extremely unpleasant, by the way
:) Oh indeed, it's not a pleasant task whatsoever and while some are relatively straightforward others will not be so and require core re-factoring to free them up to move.

• Changing selectors and the structure of markup mostly *cannot* be done without breaking compatibility.

And that would be an issue, there is little point to the exercise overall if we're not able to address issues of this type, although preserving classes/IDs could be possible, being forced to retain markup though if found not to be optimal essentially probably nullifies the exercise.

IMHO, the guiding principle should be: If it can be done in bp-legacy, it should be done in bp-legacy. If it cannot be done in bp-legacy, it should be done as part of a full rebuild that includes a redesign.

I'm not convinced that if it can be done in bp-legacy is the right way of thinking, a lot probably can be and we just end up performing a half baked exercise? A full re-build is probably the only way of tackling this, but and it's an important but, design does not dictate function, we know the saying form follows function we must have the function correct before the form is considered, the messages component in template pack was an example of how an attempt was made to create form when the function wasn't correct or ready to be 'formed'

I'll freeform one other sketchy idea, partly in thinking about how the template pack was going to be implemented - maybe we create a new repo on bp github in that we work up template files with a view to a downloadable set of files intended to be dropped into a theme to overload the standard BP templates, but that still has attendant support issues and doesn't tackle the question of markup in core.

Perhaps we only look at core markup to begin with as an isolated exercise, look to identify that, move it out to new positions in the template directories, that exercise, albeit it arduous and less than fun, has benefits to core and to developers building with BP.

[boonebgorges]

To some extent I'm playing devil's advocate here. I'm not fundamentally opposed to a complete rewrite. I just want to make sure that if we're going to cause disruption - to the core team and to site admins/devs - then we should make sure it's well worth our while.

The discussion about refactored templates would probably be much easier to have if we had something to look at. It sounds like a good idea to start another repo somewhere to sketch something out. Likewise, there's no point in completely rebuilding our JS if it's going to become obsolete due to a full rewrite of our templates, so I'd suggest that the JS teardown wait until we have new markup.

I'd suggest handling the extraction of markup from core functions as a separate task, as it can be done piecemeal, and to a large extent within the context of bp-legacy. Some specific JS improvements - like the extraction of jQuery.cookie - can also be done in standalone tickets.

[hnla]

To some extent I'm playing devil's advocate here

Keanu Reeves or just some new game you got?

The discussion about refactored templates would probably be much easier to have if we had something to look at

Well, happy to see any offerings in that direction, although this heads off down the design path again which is fair enough.

I'd suggest handling the extraction of markup from core functions as a separate task

It was grouped in as it's part of the overall notion of things that should be done for BP.

as it can be done piecemeal,

Yes it's a task that can be broken up into smallish parts, stopped and started as it were.

and to a large extent within the context of bp-legacy.

Can be entirely within bp-legacy as it's pretty much moving something from one place to another while preserving its functionality, possibly adding to the functionality but which doesn't have backpat concerns.

can also be done in standalone tickets.

Probably best as a master ticket to identify tasks then tasks can be broken out to tickets to work on also tasks could be grabbed in any order if someone fancied one to tackle.

[boonebgorges]

The discussion about refactored templates would probably be much easier to have if we had something to look at

Well, happy to see any offerings in that direction, although this heads off down the design path again which is fair enough.

No - what I mean is that I'd like to see an example of what you're thinking about re: improved markup and selectors. The discussion so far has been quite abstract. Maybe you could pick a template or two and give it your proposed treatment, and then present that to the group for feedback?

Probably best as a master ticket to identify tasks then tasks can be broken out to tickets to work on also tasks could be grabbed in any order if someone fancied one to tackle.

This is generally fine, but if the task is going to take longer than one dev cycle, we should open separate tickets, as it's nice to be able to close a ticket against the milestone in which the commits appear :)

[hnla]

The discussion so far has been quite abstract.

Yep fair comment it has been in the abstract mainly, and Paul asked for something more example-ish, just haven't had much time to produce some.

it's nice to be able to close a ticket against the milestone in which the commits appear

Good point, although a master reference does seem to make sense shame we can't have them but out of mainstream ticket flow, sort of notebook entries.

[hnla]

Core files frontend code: Identifying & extracting - where possible - all markup from core to template files, ...

Revisiting this ticket to look at instigating the point above as a preliminary series of tasks, that are standalone in their own right. but also required in advance of any possible re-building of new template files ( once moved it allows the freedom to do what one wants with the markup as it now lives in the template section of core and is easily overloadable.)

I have begun re-factoring a local install to test this process for extracting and re-positioning core markup functions.

Our primary benefits here are:

• We remove and lighten the core file with less code to maintain,. In the new file we can re-factor the functions to a degree to be leaner, re-using the basic makup rather than repeating it.

• Access to and customizing of these functions/markup is now a lot easier, while still retaining the apply_filter method.

Somewhat sadly in the initial pass it appears necessary to have to preserve the template tag names/functions as they are all unique to each component plus each has an apply_filters call so to effect a change like this we must be careful not to break templates or filtered functions

Naming of files/folders is obviously something needing discussion.

Current proposed & tested approach:

• In buddypress-functions.php I have created a new locate_includes() function passing to it a file name which is then checked for file_exists - this is a cut down version of existing locate_assets() this will allow us to check where we find the file and return a path to it.

• In bp-legacy/buddypress/ I have created a new folder _incl in which I propose locating any template tag like functions.

• As a first run I have added a new file bp-dir-search.php this file extracts all core html search functions from the various bp-*-templates.php core files. In this file I rework the search functions splitting out the actual markup to it's own function then running the existing function names to preserve backpat in possible overloaded templates along with their apply_filters and passing the unique params query string, markup tokens etc to the base markup function bp_get_dir_search_markup()

The initial test is on the dir search forms for which I'll create a ticket( against 2.6) if we want to progress this, then I'll create further tickets for the other functions such as dir filters, bulk-action filters; although I would like to list thse in some ticket so we can see at a glance what exists to be tackled and whether it has a ticket to cover it, i.e is underway

[DJPaul]

Closing most tickets related to BP-Default and BP-Legacy, since the upcoming BP-Nouveau template pack (planned for 3.0) will make these redundant.

[DJPaul]

Closing most tickets related to BP-Default and BP-Legacy, since the upcoming BP-Nouveau template pack (planned for 3.0) will make these redundant.