Perlan museum is a great place to visit while in Reykjavik, especially for kids and rainy days. You can easily spend a few hours here. All of their exhibits are awesome, especially the artificial ice cave you can visit all year around. - If you're unable to visit any other ice cave in Iceland, come here!

Plus, they also have a Northern Lights show in their planetarium included with your admission.