Reduce overall testing time of juno through changes in REST tests

[D-DePablos]

Tests currently take a long time because the rest api tests which are meant to fail (within /pkg/rest/tests/rest_test.go) take a long time to timeout.

This means that, while the rest of the test suite finishes in less than 10 seconds, all the Test(...)Fail take ~ 10s each, 13 times, incrementing this substantially and wasting time.

It would be good to have 1 timeout test with the normal rate, and the other failing tests to have a much smaller timeout or use a different method to check for failure than to wait out the full 10s.

[joshklop]

Maybe we add a Config struct to the rest package where the user can adjust parameters like timeouts, protocol (http/https), etc? Then in the test we simply configure the package to immediately timeout.

I haven't explored this much, but wanted to post it here just in case the idea is useful.

[D-DePablos]

I think this issue is a bit more involved than I expected as the timeout is set at the client level, and there is no easy place to change this. Would yoiu like to have a go with your approach @joshklop ?

[joshklop]

I can give it a try, but I may not be able to get to it for a bit.

[D-DePablos]

Scope might have changed: It would be good to use the httptest standard library exclusively, instead of relying on external dependencies

[v4lproik]

Hi guys! I was working on another PR and as I was running all the tests, I also saw that the tests within rest_test.go were very slow. I looked at it and the problem is that the http client timeout will never be taken into account as you have an hardcoded time.sleep in you code here: pkg/feeder/feeder.go:129.

You need to extract the default retry function (the loop) implemented there without breaking the current implementation used by other functions. What you could do is allowing to pass a retry function as parameter. Something like that:

func (c *Client) do(req *http.Request, v any) (*http.Response, error) {
	retry := func(req *http.Request, c *Client, err error) (*http.Response, error) {
		var res *http.Response
		for i := 0; err != nil && i < 2; i++ {
			time.Sleep(time.Second * 1)
			res, err = (*c.httpClient).Do(req)
		}
		return res, err
	}
	return c.doWithRetry(req, v, retry)
}

This would allow you to pass a retry function from the tests and modify the sleep time and so by extention the timeout time.

Happy to open a PR if you guys want.

[joshklop]

That looks awesome @v4lproik! Thank you so much. A PR would be great. If you need anything please let me know.

[v4lproik]

I decided to extend the client so it takes a retry mechanism function as parameter in NewClientWithRetryFuncForDoReq constructor. If the default constructor is being used then I initiate the retry function with the same piece of code that was hardcoded in the Do function before. Hope that helps.