"MetaFields": [
    {
      "Name": "_ExcludedCatalogEntryMarkets",
      "Type": "EnumMultiValue",
      "Data": [
        {
          "Language": "en",
          "Value": "DEFAULT"
        }
      ]
    }
]  

Thanks Mark!

So if I would need to exclude multiple markets I would by your example do something like this, correct?

"MetaFields": [
    {
      "Name": "_ExcludedCatalogEntryMarkets",
      "Type": "EnumMultiValue",
      "Data": [
        {
          "Language": "en",
          "Value": "DEFAULT"
        },
        {
          "Language": "en",
          "Value": "DEFAULT2"
        }
      ]
    }
]

As we get an catalog.xml file exported from a PIM-system that's well over 2GB we have to process this file in another way so when converted to JSON it looks like the code below. Does this mean we have to transform each field to the layout you posted previously or is there another way around this?

"MetaFields":[
{
"Name":"_ExcludedCatalogEntryMarkets",
"Type":"EnumMultiValue",
"Data":[
{
"Language":"en",
"Value":"",
"Item":[
{
"Key":"ITOPA",
"Value":"ITOPA"
}
]
},
{
"Language":"en-ie",
"Value":"",
"Item":[
{
"Key":"ITOPA",
"Value":"ITOPA"
}
]
},
{
"Language":"en-gb",
"Value":"",
"Item":[
{
"Key":"ITOPA",
"Value":"ITOPA"
}
]
},
{
"Language":"en-sg",
"Value":"",
"Item":[
{
"Key":"ITOPA",
"Value":"ITOPA"
}
]
},
{
"Language":"en-us",
"Value":"",
"Item":[
{
"Key":"ITOPA",
"Value":"ITOPA"
}
]
},
{
"Language":"en-in",
"Value":"",
"Item":[
{
"Key":"ITOPA",
"Value":"ITOPA"
}
]
}
]

Yes to the first question.

Is the catalog.xml the correct format?  You could split up the file in smaller chunks or make sure you have enough memory on the server to process the file.

The catalog.xml should be the correct format as it's no problems to import smaller portions. If we in example export just one minor category with a few products it's imported correctly via commerce manager and the ServiceApi catalog import. 

The main issue is that the file will continue to grow and will probably (just a guesstimation) end up being well over 10gb of raw xml..
We run a nightly import of the catalog with the daily changes which keeps the size down but occasionally there will be changes to all of the items in the entire catalog if a field is added.
Today these changes needs to be handled manually.

To handle this we're looking in to the possibility of reading the catalog.xml line by line and basically convert the xml to JSON and use the ServiceAPIs restful operations and post node by node and entry by entry as this would reduce the stress on the server as well as keeping the machine size down as the amount of memory needed to process xml-files as they are processed in the PIMs connector would be ridiculous....

Yes, I understand if the pim does not send smaller subsets of the data then it would be up to you to reduce the size.  Looking at the data in the response, it would seem the size would reduce greatly if the fields that are empty are not send to be processed.

Are all of the fields in the PIM sent over regradless if they are used on the website?

If you do process the file yourselves I would recommend not sending the empty fields.

We need to be able to empty fields that has previously been filled with data this is why we export empty fields as well, mainly because the PIM in question does not support versioning and therefore doesn't know what fields have modified....

No, we have a subset of fields that gets exported that are used on the website.

Back to the first question, should the "Items"-field not be used but instead use multiple "Data"-attributes for the excluded markets?

The bulk import schema is not used for the restful operations.  So any meta fields passed in must use the same format at the first example I posted as explained here

http://world.episerver.com/documentation/Items/Episerver-Service-API/Catalog-subsystem-restful-operations/Catalog-entries/#Put

Hi Mikael

You should not repeat the Data attribute for _ExcludedCatalogEntryMarkets. If you check the Catalog UI in the CMS, you can only modify the MarketFilter in the Master Language, in other words, you should repeat item element

Below is the example, we craked yesterday, and worked very well. In our example, we disable the default market, and create an Australia market as default one. 

     <MetaField>
      <Name>_ExcludedCatalogEntryMarkets</Name>
      <Type>EnumMultiValue</Type>
      <Data language="en-Au">
        <!-- Value is the market ID -->
        <Item value="NZ" />
       <Item value="UK" />
        <Item value="INTL" />
        ......
      </Data>
    </MetaField>x

To exclude multiple markets through the Service API: