Have a look at Wissant or Wimereux (you'll need to look up restaurants on the web) or try Le Channel in Calais itself ... has been good in the past.

In Boulogne Old Town we had a great meal at a restaurant called 'The Swan'. Walk into the town through the main gate and go up to the town square. There is a floral display with a large lawn on your left. Walk past the lawn and turn left at the end of it. Ahead is a street. The restaurant is about 50 metres on the right. I had the St Jaques (Scallops) last time and they were marvelous.