For anyone else suffering from the same, I got the answer to my question above. The reason page with published date updated to future datetime throws 404 is because while rendering page, it only looks at start Publish date, which if in future, means page ( or current version of the page) isn't published yet.

Whereas, with Schedule to Publish option, we don't actually change the Start Publish date. Instead we update the DelayPublishUntil date. So technically, for rendering, the current version is still published and renders while the new one is scheduled. When the job publishes the new version, that's when it modifies the Start Publish date on it. So now new changes will reflect on the page.

I was about to really favor Published Date change over Schedule to Publish option, especially because its automatic. You don't need a job to run to publish the page at a later datetime. But because of this one flaw above, i'd move against it.

I guess for brand new pages that need to be published later and for expired content that needs to be republished at a later date, the Published date change option may still work great. For existing published pages, stick to Scheduler option!

Hope this helps!