Skip to:
Content

Opened 15 months ago

Last modified 4 weeks ago

#4796 new enhancement

Use taxonomies instead of/in addition to BP Xprofile tables

Reported by: boonebgorges Owned by: boonebgorges
Milestone: Future Release Priority: normal
Severity: normal Version:
Component: XProfile Keywords:
Cc: vpundir@…, vivek@…, emzo, jonathan@…

Description

BP's xprofile tables store profile field data (and metadata about fields) in what are essentially key-value pairs. This works well when querying by user id, which is what generally happens on the user profile pages and elsewhere in BuddyPress. However, the unindexed value column means that it's basically impossible to do more complex queries - stuff like "how many users live in Wisconsin or New York, and also are less than 100 years old?". It's technically possible to make these kinds of things work - see http://wordpress.org/extend/plugins/bp-better-directories/ - but the JOINs required are very ugly and very unscalable.

Profile data is many-to-many in structure. And WordPress already has a system for many-to-many metadata: custom taxonomies. I'd like to propose that xprofile field data be stored in a taxonomy instead of a key-value table.

Benefits:

  • Exponentially faster complex queries
  • We can leverage the core taxonomy API, including caching
  • We can remove a bunch of stuff from BP

Concerns:

  • Backward compatibility with existing plugins/themes
  • Pain in the butt to port
  • Some field types do not lend themselves well to taxonomies. In particular, it would be odd to store <textarea> data (multi-line text box) in a taxonomy

I'd want for the xprofile component (at least the query class) to have pretty complete test coverage before attempting anything like this.

Obviously a long-term project and perhaps a pipe dream, but IMO it's one of the most promising areas in BP for getting rid of a custom table or two, while also giving us a ton of cool new features.

Change History (11)

comment:1 johnjamesjacoby15 months ago

Conceptually, it's definitely an improved data schema. Like we've talked about in the past, this goes into the 'picking a data storage blog ID for each component' route, which I'm less against now that switch_to_blog() has been recently improved.

It's also a chance for us to take what's broken in WordPress's taxonomy functionality, and fix it before they do. Their long-term plan is to fix the term_taxonomy_id issues, ditch the relationships table, and introduce a term_meta table. We could rebuild XProfile from the ground up like this, and then build a script to migrate the data over during the update process.

We could expand on this concept a bit more, and introduce a set of global field/data/meta tables for any component to use. We can eliminate the field groups table completely by making a "group" another type of "field."

  • bp_fields
  • bp_field_data
  • bp_field_meta

Then we can take the best of both taxonomies and profile fields, and use them for Group Component fields (rather than stuffing it all in meta) or any other component that needs arbitrary field/data/meta can access those tables in an efficient way.

This, in turn, opens up BuddyPress to having true access control, as we could ultimately query more complex datasets without needing to LEFT JOIN a thousand times, or run several uncached queries to get up-to-date data.

Very high level thoughts, but I like getting the discussion going. Thanks for opening this ticket, and I'm excited to see where it leads. Let's start dreaming up the optimal field data schema for this, and forge it into something we aren't limited by in the long term.

comment:2 sooskriszta12 months ago

  • Cc vpundir@… added

comment:3 sooskriszta11 months ago

The ability to run complex queries cheaply and scalably will be invaluable for BuddyPress's future.

For starters, it will enable proper member search #4541 which will be useful for many, many communities. In fact, it could enable BuddyPress to host some types of communities that have so far not been a core target.
e.g.

  • Meetup-type, user-group communities - if I can find everyone in the community between 18 and 34 who's interested in kite surfing, I could create a new group on kite surfing and invite them all
  • Professional listings - if I can easily find an illustrator with 5* expertise in CorelDraw in Camden, why wouldn't I use BuddyPress?
  • Churches, Volunteer organizations, Schools etc - Sure, many of them already use BuddyPress. But if I can easily find folks who are in town this weekend, then I wouldn't have to use other software/services to create and manage a list of who's going to the Habitat for Humanity work site this weekend.
  • Directories/Catalogs - This will make JJJ cringe, but a parameter-based search could make BuddyPress a contender for catalog software, e.g. I can find all movies directed by Hitchcock between 1940 and 1946? Wow! Okay, maybe I went a little bit too far out with this one. But the first 3 use-cases are legit.
Last edited 11 months ago by sooskriszta (previous) (diff)

comment:4 sooskriszta7 months ago

Hey guys, how are you thinking about this? Appropriate for 2.0?

comment:5 sooskriszta7 months ago

  • Cc vivek@… added

comment:6 emzo5 months ago

  • Cc emzo added

comment:7 jchristopher5 months ago

  • Cc jonathan@… added

comment:9 follow-up: sooskriszta2 months ago

Would this become any easier once @DJPaul has completed #5220 ?

comment:10 in reply to: ↑ 9 boonebgorges2 months ago

Replying to sooskriszta:

Would this become any easier once @DJPaul has completed #5220 ?

Yes, it'll at least be easier to build a proof of concept once that work is done.

comment:11 sooskriszta4 weeks ago

Yay! Now that #5220 is done, I hope and pray this is moved to milestone 2.1

Note: See TracTickets for help on using tickets.