The fact that hazards work OK means that the general wiring is OK.
Fuse #11 and #39 inside the cabin fuse box may be relevant but I doubt it.
Download the 2003 KJ Service Manual here....includes the wiring diagrams...close enough to the 2004 model.
You will see from the wiring diagram for the Hazard switch that the one section of the Multifunction switch namely connector C1 pin 8 is supposed to be at ground at all times...this switch toggles one way to provide this ground to pin 6 or toggles the other way to provide ground to pin 10. this causes the left side or the right side turn signals to operate.
So I would definitely say that the ground point has gone missing. So temporary splice a wire from a clean chassis point to the wire on Multifunction switch connector C1 pin 8....probably a Black/Tan wire. If it now works then your ground has gone missing.
If you leave the toggle switch in its center ie. not trying to indicate that you are turning and touch ground to pin C1 pin 6 then one side should flash and then move the ground to C1 pin 6 and the other side should flash. I am having problems at the moment attaching the relevant page of the Flasher circuit but you will find it if you download the 2003 KJ Service Manual here:
Index of /manuals/Jeep/KJ