You need an explorer.

Edit: I previously stated that you need Umbra/Underworld ambassador to actually gain exploration points, but it it no longer seems to be the case. As of now, explorers gain them points on their own, ambassador (and certain other things) just add some extra.

Interesting.. I want to learn more.

You can assign certain courtiers or yourself to the role of an explorer. Enter the umbra, pick a location that you like. Assign an explorer - there is a window for that in the lower ríghthand corner. If your character can enter that location then you can explore it. Werewolves and fae courtiers can automatically be used as explorers. Some mortal mages, wraiths and powerful vampires with the right disciplines like high level auspex could be useful as well. Not verybody can access every part of te umbra, though. More explorers in different locations --> more loot.