First of all, I think 3-4 second is too long. I am a slow shutter speed fanatic, I just love it. I've had some good experiences with 1 second.
But you do need a tripod for that kind of thing or at least a mini-pod that you can put on a rock. The rule of thumb in relation to when you need a tripod is: Any shutter speed slower than the Focal length needs a tripod. For example: If you're shooting with a tele lense on 200mm anything under 1/200 second would require a tripod. If you're shooting on 30mm you can go to 1/30 (or the nearest).
Anyway, back to the water. With the long shutter speed you let in much more light, so you need to filter out light. Neutral Density Filters are the best option for that.
There are a couple of different filter packs available and the holder I used let me put on up to 3 different filters. So if you stick on 2 ND filters and see what reading you get, you should be fine.
The main thing is to experiment with shutter speeds, it's made much easier now with digital where you can see instantly what the image looks like. But nevertheless, do an exposure series and see what effect you like best.
The picture below was taken with an exposure of 1 second, with a medium aperture (I think about F8 or F11, not sure).
Hope this helps.