You could retrieve the the original object when listing results.

If you do GetContentResult it's already in the cache.

If it's a page you could can do something like this:

public static PageReference PageLink(this UnifiedSearchHit hit)
{
    if (typeof(PageData).IsAssignableFrom(hit.OriginalObjectType))
    {
        PageReference pageReference = PageReference.ParseUrl(hit.Url);

        if (!PageReference.IsNullOrEmpty(pageReference))
        {
            return pageReference;
        }
                
        object internalObject;

        if (Global.UrlRewriteProvider.ConvertToInternal(new UrlBuilder(hit.Url), out internalObject))
        {
            pageReference = internalObject as PageReference;

            if (!PageReference.IsNullOrEmpty(pageReference))
            {
                return pageReference;
            }
        }
    }

    return PageReference.EmptyReference;
}

Thanks Johan, that's useful especially the cache advice.

Or just call the OriginalObjectGetter:

public static T GetOriginalObject<T>(this UnifiedSearchHit hit)
{
    if (hit.OriginalObjectGetter != null)
    {
        var original = hit.OriginalObjectGetter.Invoke();

        if (original is T)
        {
            return (T)original;
        }
    }

    return default(T);
}

Then you can:

var content = hit.GetOriginalObject<YourPageDataPage>();

Thank you very much!