When the game is closed, all level progress is removed, which is necessary to unlock the last level! To avoid this force close gunworld.exe from within the Processes tab of the Windows Task Manager (ALT+CTL+DEL). Doing so will retain the temporary save game data. You will still need to complete the training mission again, but the level select screen will have all previously completed missions still marked with guns and expansion pack progress remembered.
Saving has been added in V2.0!

AntiMicro[github.com] is a Free, open source program I used to map my PS2 controller to the keyboard keys. It even has a configurable turbo option to rapidly fire while the button is held down.

Press the jump key while next to a wall to perform a wall jump.

This is possibly the quirkiest piece about this game. The blue bar at the bottom indicates the amount of ammo remaining. Running out will equip the useless machete. Press the DOWN+FIRE keys to throw a seed (of the currently selected weapon), which changes into that gun upon hitting the ground. To pick it up, press FIRE while touching it. There are unlimited seeds (ammo) for the default gun (Magnum).

As long as you complete all levels in the game most of the achievements should unlock. The descriptions on most are self-explanatory. Here are any that may need additional work/explanation:

Surrender your guns and embrace peace, you sissy
The first time through you probably shot the alien priest at the end. This will unlock if you play through the entire last mission again and after the final boss fight do nothing. Let the priest finish his long narrative dialogs. Here is a video showing this ending:
www.youtube.com/watch?v=r2PH0WdN5Yk

Collect every seed expansion pack
There are 4 expansion packs of each gun (20 total) hidden throughout the levels. Once picked up, they remain with you even if you don’t finish the level and they are replenished with each life. They look like this and their color represents the weapon:

I was going to map out their locations, but got stuck on the one above within Jungle (walk left after leaving the cave.) There doesn't seem to be any way to get inside.