Import of large sites fails

November Happy Hour will be moved to Thursday December 5th.

AI OnAI Off

Home / Forums / Developers (Main Products) Forum / Content Management System 12 /

Import of large sites fails

Carl S

Vote:

We're in the process of changing from CMS 11 to CMS 12 and want to start on a fresh database.

Therefore we want to use the Import data feature, how ever, we get the error "timed out or max file size exceeded".

I've tried multiple things:

https://www.webdavsystem.com/server/documentation/large_files_iis_asp_net

web.config

<configuration>
	<system.webServer>
		<modules>
			<remove name="RequestFilteringModule"/>
		</modules>
		<handlers>
			<remove name="aspNetCore"/>
			<add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified"/>
		</handlers>
		<aspNetCore processPath=".\*.exe" stdoutLogEnabled="true" stdoutLogFile=".\logs\stdout" hostingModel="inprocess" requestTimeout="02:00:00">
			<environmentVariables>
				<environmentVariable name="ASPNETCORE_ENVIRONMENT" value="Local" />
			</environmentVariables>
		</aspNetCore>
		<httpErrors errorMode="Detailed" />
		<httpProtocol>
			<customHeaders>
				<remove name="X-Powered-By" />
				<add name="X-UA-Compatible" value="IE=Edge" />
			</customHeaders>
		</httpProtocol>
	</system.webServer>
</configuration>

Startup.cs

// If using Kestrel:
services.Configure<KestrelServerOptions>(options =>
{
	options.Limits.MaxRequestBodySize = null;
	options.Limits.KeepAliveTimeout = TimeSpan.FromMinutes(60);
	options.Limits.RequestHeadersTimeout = TimeSpan.FromMinutes(60);
	options.AllowSynchronousIO = true; // NewtonSoft.Json issue
	options.AddServerHeader = false;
});

// If using IIS:
services.Configure<IISServerOptions>(options =>
{
        options.MaxRequestBodyBufferSize = int.MaxValue;
	options.MaxRequestBodySize = null;
	options.AllowSynchronousIO = true; // NewtonSoft.Json issue
});

// uploading options for EPi
services.Configure<UploadOptions>(options => options.FileSizeLimit = 4294967295L);

Startup.cs

app.UseWhen(context => context.Request.Path.StartsWithSegments(new PathString("/EPiServer"), StringComparison.InvariantCultureIgnoreCase), builder =>
{
	var maxSizeFeature = builder.ServerFeatures.Get<IHttpMaxRequestBodySizeFeature>();
	if (maxSizeFeature is null)
	{
		return;
	}
	maxSizeFeature.MaxRequestBodySize = null;
});

#304346

Edited, Jun 30, 2023 6:01

Quan Mai

Vote:

How big is your file, and what is exactly the error message (including stacktrace if any?)

#304358

Jun 30, 2023 10:59

Carl S - Jul 03, 2023 4:46

Hi Quan,

Everything over 30MB fails. - But some of our sites is 4 GB or so.

Setting 'MaxRequestBodySize' to null is supposed to fix the issue.

No error message in logs. Only the visual message in backoffice "timed out or max file size exceeded".

/ Carl

Paul McGann (Netcel)

Vote:

Hi Carl,

This same question was asked on the forum here with and accepted answer.

Paul

#304449

Jul 02, 2023 18:05

Carl S - Jul 03, 2023 3:20

Hi Paul.

This setting only affects file uploads in the media folders.
And I've already set this by code:

// uploading options for EPi

Carl

Paul McGann (Netcel) - Jul 03, 2023 8:25

Have you tried it with just that configuration setting?

I have this in my current projects and it is working:

"EPiServer": {
"CmsUI": {
"Upload": {
"FileSizeLimit": 104857600
}
}
},

Are you running this under a linux OS?

Carl S - Jul 03, 2023 8:33

No I have not. Setting it by code should be enough. But I can try that.

No. Windows and IIS.

Carl S - Jul 03, 2023 8:38

Your suggestion to add only configuration setting did nothing. Same error.

Quan Mai

Vote:

Just to be sure, did you combine both settings (in IIS and in application layer)?

#304501

Jul 03, 2023 9:33

Carl S - Jul 03, 2023 9:36

Combine both settings? :)

Quan Mai - Jul 03, 2023 9:40

yes, basically enable the big files upload on both IIS and on application layers.

Carl S - Jul 03, 2023 9:45

I have tired multiple variations. But yes.

remove name="RequestFilteringModule" is supposed to remove the max file limit in IIS.

Have also tried requestLimits maxAllowedContentLength="4294967295" in web.config.

And you can see by my code - setting 'MaxRequestBodySize' to null is also supposed to remove the max file limit.

Quan Mai

Vote:

A shot in the dark, but maybe this in addition of your current settings

            services.Configure<FormOptions>(x =>
            {
                x.ValueLengthLimit = 5000; // Limit on individual form values
                x.MultipartBodyLengthLimit = 737280000; // Limit on form body size
                x.MultipartHeadersLengthLimit = 737280000; // Limit on form header size
            });

#304508

Jul 03, 2023 9:53

Carl S - Jul 04, 2023 4:38

Awesome. I set these to the max values and I could upload a 3 GB file.
Thank you!

Carl S

Vote:

web.config

<configuration>
    <system.web>
        <compilation debug="true" />
        <customErrors mode="Off" />
    </system.web>
    <system.webServer>
        <handlers>
            <remove name="aspNetCore"/>
            <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModuleV2" resourceType="Unspecified" />
        </handlers>
        <aspNetCore processPath=".\[NAME].exe" stdoutLogEnabled="true" stdoutLogFile=".\logs\stdout" hostingModel="inprocess" requestTimeout="02:00:00">
            <environmentVariables>
                <environmentVariable name="ASPNETCORE_ENVIRONMENT" value="Local" />
            </environmentVariables>
        </aspNetCore>
        <httpErrors errorMode="Detailed" />
        <httpProtocol>
            <customHeaders>
                <remove name="X-Powered-By" />
                <add name="X-UA-Compatible" value="IE=Edge" />
            </customHeaders>
        </httpProtocol>
        <security>
            <requestFiltering removeServerHeader="true">
                <!-- IIS settings, specifies the maximum length of content in a request in bytes, default value is 30000000 -->
                <requestLimits maxAllowedContentLength="4294967295" maxQueryString="32768" maxUrl="65536" />
            </requestFiltering>
        </security>
    </system.webServer>
</configuration>

Startup.cs

services.Configure<UploadOptions>(options => options.FileSizeLimit = 4294967295L);

// If using Kestrel:
services.Configure<KestrelServerOptions>(options =>
{
	options.Limits.MaxRequestBodySize = null;
	options.Limits.KeepAliveTimeout = TimeSpan.FromMinutes(60);
	options.Limits.RequestHeadersTimeout = TimeSpan.FromMinutes(60);
	options.AllowSynchronousIO = true;
	options.AddServerHeader = false;
});

// If using IIS:
services.Configure<IISServerOptions>(options =>
{
	options.MaxRequestBodyBufferSize = int.MaxValue;
	options.MaxRequestBodySize = null;
	options.AllowSynchronousIO = true;
});

services.Configure<FormOptions>(options =>
{
	options.ValueLengthLimit = int.MaxValue; // Limit on individual form values
	options.MultipartBodyLengthLimit = long.MaxValue; // Limit on form body size
	options.MultipartHeadersLengthLimit = int.MaxValue; // Limit on form header size
});

Startup.cs

app.UseWhen(context => context.Request.Path.StartsWithSegments(new PathString("/EPiServer"), StringComparison.InvariantCultureIgnoreCase), builder =>
{
	var maxSizeFeature = builder.ServerFeatures.Get<IHttpMaxRequestBodySizeFeature>();
	if (maxSizeFeature is null || maxSizeFeature.IsReadOnly)
	{
		return;
	}
	maxSizeFeature.MaxRequestBodySize = null;
});

#304582

Edited, Jul 04, 2023 5:00

This topic was created over six months ago and has been resolved. If you have a similar question, please create a new topic and refer to this one.