With EF v4 you can use SqlFunctions.StringConvert. There is no overload for int so you need to cast to a double or a decimal. Your code ends up looking like this:
var items = from c in contacts
        select new ListItem
        {
            Value = SqlFunctions.StringConvert((double)c.ContactId),
            Text = c.Name
        };