avatar upload and 32k folder limit

[unsalkorkmaz]

Hi,
I see buddypress creating a folder for each user in /uploads/avatars. And there is a 32k limit in a single folder if i am not wrong. So basically there will be problem on avatar uploads if we have more then 32k members right?

[boonebgorges]

Good question. I have never tested. It would probably be possible to build a workaround using symlinks, which don't count toward the 32k total.

Does WP itself have any protections for this? (Probably not, since its upload directories are date-based, and it's unlikely that anyone would blog for 32,000 years.)

[DJPaul]

I'm finding it really tricky to find any authoritative sources about this. I believe that, for modern FSs, the max number of files *per (non-root) directory* is way above 32k.

• (From memory) I think FAT32 had about ~64K max files per directory.
• Interwebs seem to suggest NTFS is into the (m|b)illions.
• HFS Plus is 2.1 billion (​http://support.apple.com/kb/HT2422?viewlocale=en_US).

[unsalkorkmaz]

The limit of sublevel-directories is 31998 due to the link count limit. ​http://en.wikipedia.org/wiki/Ext2

Most of wordpress installs are on linux and using ext2 or ext3 filesystem if i am not wrong. Basically yes, there is still 32k limit per directory. This is why wp have date based splitting imo.

What i suggest? most of people dont need a change for it but some sites that expecting more then 32k member will have problems. As i found a solution for blogs.dir directory sharding in ​http://wordpress.stackexchange.com/questions/31888/sharding-bloguploaddir i suggest to let define 'AVATARUPLOADDIR' or something. So people can split it to support more then 32k.

[boonebgorges]

We have a filter called bp_core_avatar_dir which should serve the purpose that unsalkorkmaz describes. With this filter, one could write a short script that would do the necessary sharding.

It would be nice to have some internal detection for this issue, but it'd take some work to keep it fairly low-overhead and extensible. Patches welcome. Until then, use the filter bp_core_avatar_dir.

[DJPaul]

To clarify, it's not sites with >32K users; it's sites with an even high number of users, of which >32K of those will have uploaded a custom avatar, instead of using gravatar or a default fallback image. In my experience, not everyone customises their avatar. Good call on using bp_core_avatar_dir in the interim.

[Ben90]

Has anyone built a patch/hook for this that they would be willing to publish?

[DJPaul]

No? :)

I took a quick look at the code and it looks like you'd need to change the path in several different places that aren't easily changeable, so it probably means we would need to refactor some existing code before this is as easy as dropping in a custom function.

[DJPaul]

In all the years, I have never seen this theoretical problem in the wild. I think it's sufficiently edge-case for the majority of BuddyPress sites that I'm going to close this ticket for now, until such time the issue is proven when we can look at it again.