Quoting Swept Away
If you could only choose one to spent atleast two nights, what would you choose???
lets just say you are tired of Bangkok and you want something different for atleast two days.
But you also want to go cheap, but you don't mind paying for some luxuries, if its really worth the price.
That is really two different type trips. One is more scenic and has the connection with the Bridge on the River Kwai - Death Railway attractions. Pattaya, on the other hand, is more known for cheap accommodation, interesting nightlife and probably more partying. But there are other non-party type things you can do.
Many tourists read posts from people who say "don't go to Pattaya" then the posters admit that they have never been there themselves! Pattaya is only around 2 hours from Bangkok by bus and probably around 130 baht for the ride. Kanchanaburi has lots of tours you can take right from Bangkok, some are for overnight. If you don't like Pattaya just don't put it on your "must return" list for next trip to Thailand!
So, it is up to you!