I also have a similar issue with a site that currently has around 60K content items. The customer also describes it as freezing of asset pane and no-so-great performances in edit mode.

I tried to minimize this to some extent, by hiding some of the content items in edit mode (for me, this was possible due to the purpose of a big part of the tree):

[ServiceConfiguration(typeof(IContentQuery))]
    public class GetChildrenSkipProfilesForEditorsQuery : GetChildrenQuery
    {
        public override int Rank
        {
            get
            {
                return 100;
            }
        }

        public GetChildrenSkipProfilesForEditorsQuery(IContentQueryHelper queryHelper, IContentRepository contentRepository, LanguageSelectorFactory languageSelectorFactory)
            : base(queryHelper, contentRepository, languageSelectorFactory)
        {
        }

        protected override IEnumerable<IContent> GetContent(ContentQueryParameters parameters)
        {
            var content = base.GetContent(parameters).Where(c => !(c is ProfilePage) || FilterAccess.QueryDistinctAccessEdit(c, AccessLevel.Administer));
            return content;
        }
    }

Not sure if this is possible for you, so you have a couple of queries and then they could be valid for different editor roles?

BR,

Marija