unchecked malloc failure can cause segfault

[totaam]

As it is, a malicious remote user can cause a segfault during data parsing by supplying an extremely large big number, which will cause malloc to fail and then memcpy will crash on NULL.
Here's an easy fix (two commits as I messed up the first one).
http://xpra.org/trac/changeset/12729
http://xpra.org/trac/changeset/12730

This also fixes a similar issue for realloc, this time by checking the return value - you may want to use the same approach everywhere for consistency. I guess the realloc case is less critical as applications tend to encode their own data rather than data provided by a remote user, but this can still crash.

[aresch]

Thanks for the report. I'm just now starting to look into this.

I think one way we could fix this is to limit the search for the CHR_TERM to MAX_INT_LENGTH. Since we only allow the encoding of a 64 character string for a big number, we shouldn't look further than that and throw an error if we don't find CHR_TERM.

Regarding the use of malloc, I'll have to investigate that a bit more and understand the performance implications either way although I think you're probably right.

[aresch]

This has been fixed in 92e1ebc

I did remove the malloc as it is a better solution and was actually slightly faster in my tests.

[totaam]

Thanks.
I think you forgot about the realloc case, a simple assert not NULL would do the job there.

[aresch]

Oh right. I'll take a look at that.

[aresch]

Fixed in d3c779b