Fetching and parsing SN relationships per type instead of all in a single request

[Manj75]

Hi Herve,

In relation to the issues already raised and having tried the latest v1.7 I wanted to put forward a suggestion to improve the working of the import of large data sets. It is expected that relationships data set will be the largest, even bigger than any of the CI tables and have to retrieve them in a single REST request may well be inefficient and hide problem areas.

I noticed that in v1.7 the URL generated has a query filter inbuilt to fetch relationships that are IN the given lists of type sys_ids. My suggesstion is to instead of a single URL request for all relationship types, can you instead iterate the given types in the ini file and generate and send a URL REST request per type, so the JSON response is only in context to that type parsing and adding to the Archi model. I see this as being much more efficient as the JSON response will be smaller and if there is a particular failure due to a specific relationship type or size of response then it is clearly seen in the log file as progress will have been shown up to that types' request/import.

Hopefully, you can follow what I mean here, but here is some pseudo description:

for N iterate through relationship types specified in ini file
{
Generate URL request for N type
MyConnection send the request and receive JSON response
Parse the resopnse as usual and create Archi elements
Move on to next type in list
}

Thanks,
Manjit

[herve91]

I keep you request, but please note that it won't be soon.