First find out exactly why the site is slow.

If it's only slow in production look into if you can install New Relic or Red Gate ANTS in the enironment to get a good look inside. SQL Server Profiler might also be of help.

In development I like to use dotTrace and SQL Server Profiler to find out if something is doing something bad or unnecessary.

Normally it's possible to pinpoint certain data calls and cache those objects instead of going to output caching where they're used.

Output Caching will add another layer of "must think of this" and make it more difficult to use personalisation stuff like Visitor Groups or even something as easy as just setting a cookie will take some planning.

In my experience it's often integrations with other systems or other remote calls that's taking up too much time in the visitor's "thread", try to avoid this and fetch this data async somehow instead.

When you cache data by "first request" it's important to implement cache locks to avoid cache races, the best answer here shows a good way:
http://stackoverflow.com/questions/5578744/doing-locking-in-asp-net-correctly

Joel has a good post on integration patterns: http://joelabrahamsson.com/episerver-integration-patterns/