Hmmm... Restart site will actually touch web.config and that in turn will recycle the app pool. If you're in a dev env you need to make sure you have web.config checked out. But I think you will get an error unless it is writable... Strange.

Restart button should restart the site (obviously) and it should be enough.

Could you provide more details from your YSOD screen? 

In addition you can enable Debug logging for add-on system by adding following logger and appender in your EPiServerLog.config:

<?xml version="1.0" encoding="utf-8"?>
<log4net>
  <appender name="debugPackagingFileLogAppender" type="log4net.Appender.RollingFileAppender" >
    <file value="App_Data\EPiServer.Packaging.Debug.log" />
    <encoding value="utf-8" />
    <staticLogFileName value="true"/>
    <datePattern value=".yyyyMMdd.'log'" />
    <rollingStyle value="Date" />
    <threshold value="Debug" />
    <lockingModel type="log4net.Appender.FileAppender+MinimalLock" />
    <appendToFile value="true" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %level %logger: %message%n" />
    </layout>
  </appender>

  <logger name="EPiServer.Packaging" additivity="false">
    <level value="Debug" />
    <appender-ref ref="debugPackagingFileLogAppender" />
  </logger>

</log4net>

Also you can use Firebug/Chrome dev tools to inspect network calls and get more information about failed requests if any.

Ah, found the cause of the problem. The modify rights for the IIS_IUSRS group had been removed. Added Modify rights to the web.config file and it worked like a charm again.

Thanks for the hint :)