Most annoying thing is that there's no flexibility in the address book. You can't sort the viewing order by e.g. the Transport column. So if you have thousands of clients in your clients contact folder, and have this as an OAB, and these people have fax numbers listed, you get shown each person twice. One for SMTP and one for FAX. If you then try to select them all and BCC them, you get failures because it wants to send to them via FAX. Ideally you should be able to sort by transport, so you have all the SMTP at the top and FAX at the bottom, or vise versa, then you can just select the SMTP lot and do your mailing.
Phone number formatting is also a ballache. Brackets, leading zeros.. etc.