Episerver.Find 13.4.4

I'm looking at a search page with some filters. I've got four categories with filters, and between each category there is AND. Within the same category, there is OR.

For the query, we're using FilterBuilder with Or inside each category to create a custom filter, and then applies all filters to the query.
The problem is the facet counts. Using TermsFacetFor() the count will be correct if 0 or 1 filter is selected within each of the four categories (boxes).

The problem accours if I select more than one filter inside the same category(box), as the count is calulated as if the filter where AND inside each category instead of OR.

Example, inside the first category, I have three filters A, B, C
Filter A: returns 3 pages
Filter B: returns 3 other pages
Filter C: returns 3 other pages (not the same as B and C)

If I search without any filters, the facet counts will be:
- A = 3
- B = 3
- C = 3

If I select A, the facet count will be (as if they where AND, not OR)
- A = 3
- B = 0
- C = 0

But what I want is
- A = 3
- B = 3
- C = 3


What would be the most elegant way of achieving this?

My initial thought, is performing four queries, one for each filter category, just to get the facet counts.

For each category:
- filter on the other three categories
- termsFacetFor this category
- get the facets count


...this works, but it seems so very very wrong! How can I make this right?!