My usual suggestion would be Marburg - beautiful little town with a castle, the famous Elisabeth church and lots of half-timbered houses. Definitely worth a visit. About 1 hour 20 minutes from Darmstadt.
http://www.marburg.de/en/24679
http://www.panoramio.com/map/#lt=50.814788&ln=8.769239&z=4&k=2&a=1
Also, even though a lot of people don't like it, Kassel is a great town and is definitely worth a visit, especially for its mountain park Wilhelmshöhe, the castle of the same name, the Löwenburg and the Herkules as well as the parks surrounding the river Fulda. The inner city was completely bombed in WW2 and only a few buildings remain as reminders of what it used to look like so it doesn't really stand out, but as soon as you leave the inner city, you can tell there is still a lot to see. It hosts the international arts exhibition documenta every five years and a lot of the exhibits remain in the city. About 2 hours drive from Darmstadt.
http://www.kassel-tourist.info/cms03/startseite.en/uebersicht/index.php
Some other nice towns in Hessen:
Amöneburg which is just 10 minutes by car from Marburg, a beautiful tiny town situated on top of a volcano with stunning views all over the surrounding valley.
Fritzlar, around 2 hours from Darmstadt in the same direction as Kassel, is also a nice small medieval town with a fantastic cathedral.
Grebenstein near Kassel, around 2 and a half hours from Darmstadt, still has a lot of the medieval walls and towers and is also really nice, especially if you have a tour guide as there is an incredible lot of history for such a small town. Probably only worth it if you're already near Kassel or doing a day trip to Fritzlar and Grebenstein.