With EF v4 you can use SqlFunctions.StringConvert. There is no overload for int so you need to cast to a double or a decimal. Your code ends up looking like this:
var items = from c in contacts
select new ListItem
{
Value = SqlFunctions.StringConvert((double)c.ContactId),
Text = c.Name
};