Issues in Optimizely when creating large number of users programmatically

Vote:
 

Hi, We are trying to create around 100k users in Optimizely programmatically but the issue is can't able to run _userManager.CreateAsync method parallelly as we are facing the following errors with multiple threads.

           1. System.InvalidOperationException: An attempt was made to use the context while it is being configured. A DbContext instance cannot be used inside 'OnConfiguring' since it is still being configured at this point. This can happen if a second operation is started on this context before a previous operation completed. Any instance members are not guaranteed to be thread-safe.

             2. A second operation was started on this context before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext.

and when to run in series the process is becoming slow with the increase in the number of users migrated and I can't able dispose and instantiate ApplicationUserManager due to issues with DBContext. Can somebody please help me with possible ways of migrating a large number of users in to optimizely?

#285739
Edited, Aug 18, 2022 11:22
Vote:
 

Hello, 

Here are a couple of tips that will hopefully help you 😀

  1. Ensure any async calls are awaited and that a Task is always returned (not void).
  2. In order to execute CreateAsync in parallel the UserManager cannot use the same DbContext instance. A new context can be provided when instantiating the UserManager, for example:
    var userManager = new UserManager<ApplicationUser>(new UserStore<ApplicationUser>(new ApplicationDbContext()));
    The main point is that in every thread the user manager should to use a new DbContext, preventing issues such as 'A second operation was started'. It should be possible to do this using dependency injection as well.
  3. The UserManager will auto save your changes. You can possibly turn this off and bulk save the newly created users for better performane. See this post on how to do that. 
#285790
Aug 19, 2022 0:29
Vote:
 

Hi ynze, thanks for the response

currently I am getting the user manager in scheduled job like this.

var userManager = ServiceLocator.Current.GetInstance<ApplicationUserManager<SiteUser>>();

when I changed this to

var userManager = new UserManager<ApplicationUser>(new UserStore<ApplicationUser>(new ApplicationDbContext()));

I am facing two issues

At UserManager:  There is no argument given that corresponds to the required formal parameter 'optionsAccessor' of 'UserManager<ApplicationUser>.UserManager(IUserStore<ApplicationUser>, IOptions<IdentityOptions>, IPasswordHasher<ApplicationUser>, IEnumerable<IUserValidator<ApplicationUser>>, IEnumerable<IPasswordValidator<ApplicationUser>>, ILookupNormalizer, IdentityErrorDescriber, IServiceProvider, ILogger<UserManager<ApplicationUser>>)

At ApplicationDbContext: I got the following errors

1. Using the generic type 'ApplicationDbContext<TUser>' requires 1 type arguments.

There is no argument given that corresponds to the required formal parameter 'options' of 'ApplicationDbContext<ApplicationUser>.ApplicationDbContext(DbContextOptions<ApplicationDbContext<ApplicationUser>>)

Currently modified as below but still facing issue with UserManager regarding 'optionsAccessor' parameter

DbContextOptions<ApplicationDbContext<ApplicationUser>> options = new();
var userManager = new UserManager<ApplicationUser>(new UserStore<ApplicationUser>(new ApplicationDbContext<ApplicationUser>(options)));

Can you please help me with these parameters which are required in the instantiation of the user manager

#285796
Edited, Aug 19, 2022 7:13
Vote:
 

why do you need to run in parallel?  If is 100K users one time I would just create a schedule job and do synchronously

#285808
Aug 19, 2022 16:13
Vote:
 

+1 to Mark.

await immediately CreateAsync one at a time.

#285813
Aug 19, 2022 20:17
Vote:
 

As a rule of thumb, always try to max out single thread performance before thinking about running in parallel. DotTrace is a great tool to find bottleneck and optimize .

#285889
Aug 21, 2022 7:53
Vote:
 

I am handling this migration in scheduled job only. The issue with importing users synchronously is that the process is slowing down with the number of migrated users. It completed only 16000 users in 4hrs and I am trying to speed up this process.

#285932
Aug 22, 2022 5:54
Vote:
 

As I said above, try dotTrace to see if you can find the bottlenecks in your code

#285940
Aug 22, 2022 8:28
Vote:
 

Praveen. I've just written this quick snippet. Try this and amend to your own needs if it works. Remember to call wait() on your entire method of logic inside the Execute() of the schedule job if you want the job not to finish before your async methods do.

                foreach (var user in userList)
                {
                    // Create new dbcontext for every user
                    var _configuration = ServiceLocator.Current.GetInstance<IConfiguration>();
                    var options = new DbContextOptionsBuilder<ApplicationDbContext<SiteUser>>().UseSqlServer(_configuration.GetConnectionString("EcfSqlConnection")).Options;
                    using (var dbContext = new ApplicationDbContext<SiteUser>(options))
                    {
                        var store = new UserStore<SiteUser>(dbContext);
                        var result = Task.Run(() => store.CreateAsync(user)).Result;
                    }
                }

#285945
Aug 22, 2022 9:09
Vote:
 

Thanks, Surjit for the response. Very much helpful

#285949
Aug 22, 2022 10:49
* You are NOT allowed to include any hyperlinks in the post because your account hasn't associated to your company. User profile should be updated.