Pangkor is a really nice island. There should be a Transnational (I think it's spelt Transnasional though) bus from Ipoh to Lemut. The ferry from Lemut is fine, it's a catamaran style ferry so very safe...don't get off at the first stop though, get off at the second stop!
Accommodation wise, the most popular place when we were there in January on Teluk Nipah beach seemed to be the Nazri Nipah huts which has both shared bathroom and private bathroom huts. We stayed at Nipa Nipa (near the entrance to Teluk Nipah village set back from the main road) which was MYR60 for a double room with air con, hot water and TV and DVD player.
If you arrive on a weekend you might find it difficult to get reasonably priced accommodation as it gets lots of weekenders from KL there, so try to arrive Sunday - Thursday. You can hire mountain bikes to cycle around the island, there are kayaks for rent and fishing/snorkelling trips to keep you busy.
Best restaurant we found was Horizon, good Chinese and Malay food and quite cheap too with a range of portion sizes.
Oh and beware of the monkeys, if you are on the beach they tend to steal your food if you leave any lying around - even from a half open rucksack...cunning little things!
[ Edit: Edited on Nov 30, 2008, at 5:27 AM by loubylou ]