How's running tests sequentially a quick hack? I think it's a valid solution.
Parallel execution of tests is so baked into test runs that xUnit doesn't even have flag to run tests in series - you walkaround it by putting tests into one collection.
Not being able to run tests in parallel means that your code is probably not thread safe, and it is considered a major red flag.
OK that's new to me, I have been not using xUnit for quite a long time, and i assume they shifted to parallel execution in later version.
What you are trying to say, I suppose, is that TimeMeters.Start and AssemblyScanner.HasEPiServerFrameworkDependency are not thread safe. they aren't. however if they should be thread safe or not, is quite arguable. they are meant to be used by initialization process so there should be no multi threading involved.
which kind of tests you are trying to run? Why does it need to initialize the framework?
As i be live i've written i'm using mvc testkit that runs host:
[Theory]
[InlineData("/")]
public async Task PageReturns200(string url)
{
var suite = new HttpTestWorker();
var response = await suite.HttpClient.GetAsync(url);
Assert.Equal(System.Net.HttpStatusCode.OK, response.StatusCode);
}
TestWorker:
protected override IHost _host { get; }
private HttpClient _httpClient;
public HttpClient HttpClient => _httpClient;
private HttpTestWorker()
{
var factory = new CustomWebApplicationFactory<Program>(Register);
_httpClient = factory.CreateClient();
_host = factory.GetHost();
AddDefaultMocks();
}
protected void Register(IServiceCollection services)
{
services.ConfigureBasicMocks();
ReplaceOverride(services);
}
public static void ConfigureBasicMocks(this IServiceCollection services)
{
MockDatabase(services);
RemoveHostedServices(services);
}
private static void MockDatabase(IServiceCollection services)
{
var options = services.Where(x => x.ServiceType.Name.Contains("DbContextOptions")).ToList();
foreach (var service in options)
{
services.Remove(service);
}
var optionsBuilder = new DbContextOptionsBuilder<KpiDatabaseContext>();
optionsBuilder.UseInMemoryDatabase("kpiDatabase");
services.AddSingleton(optionsBuilder.Options);
var optionsBuilder2 = new DbContextOptionsBuilder<EPiServer.Marketing.Testing.Dal.DatabaseContext>();
optionsBuilder2.UseInMemoryDatabase("kpiDatabase");
services.AddSingleton(optionsBuilder2.Options);
var optionsBuilder3 = new DbContextOptionsBuilder();
optionsBuilder3.UseInMemoryDatabase("epi");
services.AddSingleton(optionsBuilder3.Options);
}
and from mvc test kit:
public class CustomWebApplicationFactory<TProgram> : WebApplicationFactory<TProgram> where TProgram : class
also currently overriding dbcontextoptions doesn't seem to work as expected
So why i'm doing initialziation? - because i'm running whole host inmemory.
Why i'm running entire host inmemory? - because epi is a minefield of extension methods with service locators in them and other traps like "content area", that makes unit tests as joyful as prostate exam.
Hi
It seems that attempting to run otpimizely solution in parallel using aspnetCore.Mvc.Testing and its webApplicationFactory to run component tests results in exceptions:
It is random which exception i will get but it seems that there is static list of something somewhere inside.
For now walkaround is to run tests sequentially but this is more a quick hack than a solution.