Liberty City is filled with various weapons scattered in shady, mysterious places... these spots turn out to be quite useful sometimes, you can get some weapons you don't have access to in the storyline yet, you don't have to spend any dollar or you can just get there if you're too lazy to go to the nearest Ammunation store.
This guide only shows the most useful spots there are on the map. This is not a full list of every hidden weapon pickups!

There is a long sloped alley you can find at the South-East of Saint Mark. It is connected to numerous small yards(2) in which you can find a hidden package along with an Uzi, giving you a fair amount of defense. You can pick-up this Uzi anytime if you're on your way to Toni's or Salvatore's missions.
Find a tall yellow building with an arch(1), that's where you should see the alley.

This spot is really close to the previous one (the Uzi), and gives a more powerful gun : the AK-47. This is the best weapon you'll be able to get on Portland, and it also comes with some protection!
Find a tall building connected to two sloped roads going from West to East, there's a Mafia Sentinel parked in front of this building(1).
There's a small alley on the right of the building (with a heart pickup), follow it until you're in a dead end behind Toni's restaurant. The AK-47 will be there with a kevlar, giving you 100 Protection Points(2).

This shotgun can be easily found near your hideout. It can be really useful for early missions.
Follow the road going to Harwood at the West of the payphone, you should see a thin passage between the road and the appartments(1).
Take it and look on the right, that's where the shotgun will be(2).

Flames in GTA 3 can burn through walls, so having a flamethrower early in the game turns out to be extremely useful! This weapon comes with a hidden package, it is found on the top of the Head Radio building.
Find the stairs leading to the aerial railway (between your hideout and Saint Mark(1)), go up to the train station and follow the rails to the North. After a small walk, you should see the rooftop with the flamethrower and the package if you didn't take it(2). Don't miss your jump, or you won't get the flamethrower!

You can find an AK-47 with a kevlar on the long boulevard of Bedford Point, look for a tall white building close to the church. There's a small set of stairs, along with a passage on the left of the building(1). At the end of this passage, you'll find both pickups(2).

This one is probably one of the most useful spot there's in the game. The M-16 isn't available to the player until you finish Arms Shortage (Ray's second mission), meaning you can get the best rifle in the game for early Staunton missions (or Portland if you use tricks to get on the 2nd island), which can be really helpful.
Go outside of the Liberty Tree parking on Bedford Point, and find a short alley with an underground parking entrance and stairs(1). Go upstairs, and get on the concrete rooftop you have on the left (there should be a hidden package on it). You also should be able to see a small yard surrounded by tall walls. Jump over the wall, you'll land on a ramp which you can use to get out of this yard. On every side of the ramp, you'll have the kevlar and the M-16 waiting here(2).

This spot is quite tricky to get to and you'll need a fast vehicle to reach it (at least as fast as a Kuruma). Follow the highway at the West of Staunton Island until you are on the spiral road leading to Shoreside Vale. At the North of this road, there's a rooftop with the sniper and the kevlar. You will need to use the white ramps of the road to jump to the rooftop and get the sniper(1). If you're lucky, you'll keep the car or it'll just flip and explode (the roof is wide enough to not take any damage from the explosion).

This one is way easier to reach than the previous one. It is found at the South of Staunton's hospital, cross the road and you'll be at the top of some stairs. The molotovs will be on the left, on a small balcony(1). If you take the other side and continue on the right, you'll have a kevlar on an other balcony.
This spot is useful to get some defense after a death!

This is the 2nd and last spot where you can have a free flamethrower (or you'll have to go to Shoreside's Ammunition if you want to buy one), cross the lift-bridge going from Staunton to Shoreside, drive down the big turn after the bridge, and go off-road on the grass. The flamethrower is located beneath the road where you took the turn(1).

If you already completed the GTA III storyline, you probably know this spot because it turns out to be really effective for one special mission : The Exchange. Go to the entrance of the Cochrane Dam (the lower part(1)), continue on the path and stop a few dozens of meters before the entrance of the container bay. There should be a small grass pad on your right, the sniper will be sitting there between two trees(2).

If you already went to Cochrane Dam for the previous spot, leave the dam and drive to the North until you come across the intersection that connects both parts of the dam and Cedar Grove. Go to the East, where you'll have a long road with a large number of houses. If you're coming from the intersection, take the 3rd house (or you can use the squares you see on the map, in this case it's the 4th square). The M-16 will be on the door step(1).

This spot is really close to the previous one, but is a bit harder to reach.
Use the same intersection that I described on the M-16 spot and take the first house you see (also first square on the radar(1)), the RPG is on the roof of the house.
Get behind the garage, you can use the small "cliff" of grass there to jump on the roof, this may take multiple tries(2).
Once you're on the roof, get to the other side by using the back of the house. Then when you're there go on the right, the Rocket Launcher will be yours, giving you 3 rockets(3).