I can get my mouse 4th button to map to delete in object mode but not in edit mode. I also don't see an edit mode keyboard configuration section in Blender preferences menu.
Open up the User Preferences, and head to the Input tab. Open the 3D View section, and inside of that the Mesh section. Now scroll to the bottom of the Mesh section and you will find an Add New button.
Now what to put in the Identifier field?
If you want the 4th mouse button to simply bring up the menu (like pressing X or Delete) then use
wm.call_menu as the Identifier (top field) and put
VIEW3D_MT_edit_mesh_delete in the name filed that will appear at the bottom. Yes it will be grayed out, but you can still click in it and add the text.
Or if you want to be able to directly delete, say vertices, by pressing the 4th button, then set up a shortcut like this:
Now the shortcut is calling
mesh.delete directly instead of calling for the menu. Then just pick which Type you want to delete. You'll notice that I have "Ctrl" checked there. This way I can press the 4th button to open up the delete menu. Yet I still can directly delete vertices, edges, and faces by holding CtrlAltShift respectively.
You have to add more shortcut entries for each "type" you want to delete.