Although Google is showing usually pictures from Grotagjá - this is different cave. You need to find the crack and there is rope to go down in one place and ladder in the other

Stóragjá is a hidden gem in North Iceland, tucked away near Reykjahlíð. This unique rift features a cave filled with naturally-heated geothermal water, but beware - the beautiful blue color comes from dangerous algae and unhealthy bacteria levels. While it may not be suitable for bathing, Stóragjá is definitely worth a visit for those looking to explore off-the-beaten-path attractions in Iceland.