One thing suggested in another thread here is adding this to your Startup.cs:

            services.Configure<FormOptions>(x =>
            {
                x.MultipartBodyLengthLimit = 10737418240;
            });

Another possibility (also from that thread) is setting the max size for kestrel:

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .ConfigureWebHostDefaults(webBuilder =>
        {
            webBuilder.ConfigureKestrel((context, options) =>
            {
                // Handle requests up to 50 MB
                options.Limits.MaxRequestBodySize = 52428800;
            })
            .UseStartup<Startup>();
        });

Did you happend to find a solution to this issue? Im having the same problem with forms version 5.5.1.

Edit: Solution found.

In the EpiserverForms package the max file size is determined by this (decompiled) method:

 private int GetmaxSizeInByte(int fileUploadsize)
    {
      int num = (int) ServiceLocator.Current.GetInstance<IHttpContextAccessor>().HttpContext.Features.Get<IHttpMaxRequestBodySizeFeature>().MaxRequestBodySize.Value;
      int multipartBodyLengthLimit = (int) this._formsOptions.MultipartBodyLengthLimit;
      return ((IEnumerable<int>) new int[3]
      {
        fileUploadsize,
        num,
        multipartBodyLengthLimit
      }).Min();
  }