So i looked around a bit in the ini files to see if we could turn down the AA on hairworks, for those of you who don't know it is set to 8x MSAA by default on all hairworks hair, this is the main reason for the extreme FPS cost.
If you go to your game install folder > Bin > Config > Base > Rendering.ini
Find the line HairWorksAALevel=8
Change it to 4, 2 or 0. Each will give more FPS. You will still get a pretty big FPS drop in dialogues, but I can live with 40 FPS dialogues.
At 0 I definitely feel its too jaggy, but 2 is fine for me and boosted my FPS when in white orchard village with the griffon trophy from 40-45 to 55-60. (Which is where I noticed my FPS took the greatest dip with hairworks)
This is with a 980 and I7 4790k.
the FPS difference between 0 and 2 isn't that big but theres definitely frames to gain there as well if you don't mind the jaggyness of the hair at 0.
Enjoy your FPS.