The visitor group impersonation works by appending a query parameter with either "visitorgroupsByName" or "visitorgroupsByID" to the request (values should be a '|' separated string with names or ids of the visitor groups). Note also for the impersonation to take effect the user needs more than just Read access (e.g. Change or Publish) to avoid that any visitor could just add the query parameters.

You can see this in the browser if you preview a page in edit mode with visitor group impersonation by looking at the url for the preview frame. 

Thank you Johan, I tried to add the querystring ?visitorgroupsByID=0a77fa7c-b06f-4ef5-bb01-656c90c81410 to the ajax request, but the method IsInRole still returns false.

When I add the querystring to the page request the method IsInRole returns true, so it works on the page request but not on the api request.

I looked a bit at the code and there is an MVC attribute VisitorGroupImpersonationAttribute that is added to ContentControllers (the base class for controllers serving content) and that does the impersonation. Now your web api does not inherit that base MVC controller. It does not contain that much code so you could write your own global filter (using Web API and not MVC) and decorate your web api controller with the attribute. The code for the MVC filter attribute looks like:

 public override void OnActionExecuting(ActionExecutingContext filterContext)
        {
            var routeHelper = filterContext.HttpContext.GetService<IContentRouteHelper>();
            if (routeHelper.Content != null)
            {
                filterContext.HttpContext.SetupVisitorGroupImpersonation(routeHelper.Content, AccessLevel.Read);
            }
            base.OnActionExecuting(filterContext);
        }