Here's what you need to do to get the the permalinks working properly:

1. Delete the .htaccess file that exists in the directory (the root directory of your dolphin installation). For me this was /public_html/

2. Upload the .htaccess file that comes with the dolphin package.

3. Reload your site.

It took me 2 hours to figure out so I hope it saves someone.