You can use the CustomerAddress for adding adresses to a contact. OrderAdress is used in the context of orders and can be created from a CustomerAddress. So the warning probably comes from using the OrderAddres with the CustomerContact.

Thank you for the answer, Jeroen.

I'm also using IOrderAddress in another context: shipment.ShippingAddress.

So, what's the proper way to set shipment.ShippingAddress property? ShippingAddress is of IOrderAddress type, so I assume we should map CustomerAddress to IOrderAddress - is that right?

Proper way is to create an address by injecting IOrderGroupFactory and then calling CreateOrderAddress.

IOrderAddress orderAddress = orderGroupFactory.CreateOrderAddress(cart)

Map your customer address to the orderAddress object.

Then add the address to your shipment on your cart.

shipment.ShippingAddress = orderAddress

The general advice is to use the new Order API (i.e. interfaces, for example IOrderAddress) instead of the concrete types, like OrderAddress. But that's not an absolute rule 

I'm not quite sure why you can use an OrderAddress for contact.AddContactAddress because there is no implicit conversion between those two. You can however write a simple method to convert an IOrderAddress to a CustomerAddress, like this (untested by me)

        public static void CopyOrderAddressToCustomerAddress(IOrderAddress orderAddress, AddressEntity customerAddress)
        {
            customerAddress.Name = orderAddress.Id;
            customerAddress.City = orderAddress.City;
            customerAddress.CountryCode = orderAddress.CountryCode;
            customerAddress.CountryName = orderAddress.CountryName;
            customerAddress.DaytimePhoneNumber = orderAddress.DaytimePhoneNumber;
            customerAddress.Email = orderAddress.Email;
            customerAddress.EveningPhoneNumber = orderAddress.EveningPhoneNumber;
            customerAddress.FirstName = orderAddress.FirstName;
            customerAddress.LastName = orderAddress.LastName;
            customerAddress.Line1 = orderAddress.Line1;
            customerAddress.Line2 = orderAddress.Line2;
            customerAddress.PostalCode = orderAddress.PostalCode;
            customerAddress.RegionName = orderAddress.RegionName;
            customerAddress.RegionCode = orderAddress.RegionCode;
            customerAddress.State = orderAddress.RegionName;
        }