So I'mma catalog all the secrets I've found so far
Keep in mind I know absolutely nothing about Stardew

Traitor Traps:
The red disk in the middle - lava trap (might want to make the ammo actually obtainable so that there's an actual reason to go there)
Far-off wall - traitor room, warps to the other 2 traitor rooms
House w/ locked door - traitor room, one-way window that can also be opened
Saloon attic - traitor room (not sure how to get into the attic without the warp from the first traitor room)
Starfruit thingy - explodes
Traitor tester - remotely activate the tester's kill button

Secrets:
Dog house - nearby note hints to put a tire over the fence, which opens up part of the fence
- button in water bowl, reveals ladder to attic in nearby house
Store - button on top of shelves (jump on the door handle, then the top of the door to reach it) unlocks back room, which hints about the iridium bars
Saloon attic - do ???, breaks attic trapdoor
Iridium bar #1 - in one of the houses, on top of bookshelf
Iridium bar #2 - in attic of house, need to activate dog bowl button first
Iridium bar #3 - in saloon attic, need to ??? first
Tree hole #1 - in graveyard, stump with opening on top, opens hole #2
Tree hole #2 - Near spawn area, tree closest to cobblestone w/ opening on side, unlocks hole #3
Tree hole #3 - Right next to far-off wall traitor room, tree w/ opening on backside, unlocks sewer
Sewer - Complete the iridium-in-tree-holes puzzle, inside is traitor tester

Hope this helps