How to reproduce

• Set up a new cms by doing dotnet new epi-alloy-mvc
• Update to latest version of Episerver.CMS (12.22.8)
• Add latest version of EPiServer.ContentDeliveryAPI.CMS (3.9.0)
  and configure Delivery API like this:
   services.AddContentDeliveryApi(options =>
 {
 }).WithFriendlyUrl();
 services.Configure<EPiServer.ContentApi.Core.Configuration.ContentApiOptions>(options =>
 {
     options.IncludeEmptyContentProperties = false;
     options.IncludeMasterLanguage = false;
 });
• Configure edtiting on external site like this
    services.ConfigureForExternalTemplates();
  services.Configure<ExternalApplicationOptions>(options => options.OptimizeForDelivery = true);
• Add a new host under Manage Websites in admin mode
  Host name: my-frontend-host.com
  Type: Primary

Errors

1. ForceAbsolute option for ContentDelivery API will have no effect on the json returned from the API. Host name is always included.
2. Thumbnail images are broken in edit mode, since they are requested using the frontend host name (but with the port of the CMS host)

When decompiling the method seems it does the following

 public void ConfigureForExternalTemplates()
        {
            ServiceLocator.Current.GetInstance<RoutingOptions>().UsePrimaryHostForOutgoingUrls = true;
            RegisterFakeTemplate = true;
        }

Based on that code seems to be doing what you are describing.  Not sure why you would ever use primary host for images though.

Did you find a solution to this? I'm facing the same issue.

Yes. This issue has "un-officially" been confirmed as a bug, but will most likely not be fixed until next major release. With the help from Optimizely I added this Initialization module. It adds an event that fixes the hostname for images, both in edit mode, and in the Content Delivery API.

[ModuleDependency(typeof(InitializationModule))]
public class MediaUrlGeneratorInitialization : IConfigurableModule
{
    private IContentUrlGeneratorEvents _contentUrlGeneratorEvents = null!;
    private IContentLoader _contentLoader = null!;

    public void ConfigureContainer(ServiceConfigurationContext context)
    {
    }

    public void Initialize(InitializationEngine context)
    {
        _contentLoader = context.Locate.Advanced.GetInstance<IContentLoader>();
        _contentUrlGeneratorEvents = context.Locate.Advanced.GetInstance<IContentUrlGeneratorEvents>();
        _contentUrlGeneratorEvents.GeneratedUrl += ContentUrlGeneratorEvents_GeneratedUrl;
    }

    public void Uninitialize(InitializationEngine context)
    {
        _contentUrlGeneratorEvents.GeneratedUrl -= ContentUrlGeneratorEvents_GeneratedUrl;
    }

    private void ContentUrlGeneratorEvents_GeneratedUrl(object? sender, UrlGeneratorEventArgs args)
    {
        if (args.Context.Host is null)
        {
            return;
        }

        if (args.Context.Url.IsPathRelative)
        {
            return;
        }

        var hosts = args.Context.Host.Site?.Hosts;
        var editHost = hosts.FirstOfType(HostDefinitionType.Edit);
        if (editHost is null)
        {
            return;
        }

        var contentLink = args.Context.ContentLink;
        if (ContentReference.IsNullOrEmpty(contentLink))
        {
            return;
        }

        var content = _contentLoader.Get<IContent>(contentLink);
        if (content is IContentMedia)
        {
            args.Context.Url.Host = editHost.Url.Host;
            args.Context.Url.Port = editHost.Url.Port;
            args.Context.Url.Scheme = editHost.Url.Scheme;
            args.IsCacheable = false;
        }
    }
}