to_string causes crazy memory usage problems

[thestinger]

5u32.to_string().byte_capacity() -> 128
"foo".to_string().byte_capacity() -> 128

I think the right solution to this is adding size hints to the underlying formatting system.

[liigo]

String is a buffer, later appends will benefits from it. If don't need a
buffer, why not use &str?

[huonw]

@liigo it needs to be dynamic constructed (i.e. &str is completely inappropriate for this), and 128 bytes is way way way too much when there is only 1 or 3 bytes actually used (like in the two examples).

[pnkfelix]

Just so it does not get lost in the shuffle, here is my current half-baked idea to address this:

• a separate trait ToStringSupport : fmt::Show { fn size_hint(&self) -> Option<uint> { None } }, and then,
• change the generic impl to impl<T: ToStringSupport> ToString for T (and then use write! into a buffer of appropriate capacity instead of format!)

see related bit of discussion here: https://botbot.me/mozilla/rust-internals/2014-08-11/?msg=19511520&page=2

It would put a minor road-block before you could call to_string() on an arbitrary type implementing Show, but it seems like a pretty simple way to get most of what is desired here.

[eddyb]

@pnkfelix adding the hint method to Show as a default method would break no existing code, I would prefer that.

[pczarn]

Good ideas. Should we use the upper bound for all integers? What about different bases? I think that integers shouldn't implement size_hint, because leading_zeros (Int trait) and size_of are sufficient for determining the number of digits in all radixes.

[huonw]

The size_hint could take a &fmt::Formatter as well as self to be able to answer the question more precisely.

I think that integers shouldn't implement size_hint, because leading_zeros (Int trait) and size_of are sufficient for determining the number of digits in all radixes.

How will they convey the computed number of digits if they don't implement that trait and method?

[pczarn]

I've just found out that Show is implemented for RadixFmt<T, Radix>, not directly on integers. This resolves my doubts.

How will they convey the computed number of digits if they don't implement that trait and method?

By using either size_hint or leading_zeros. Looks like it was a bad idea. Or by implementing size_hint for T: Show + Int.

[huonw]

not directly on integers

It is.

[pczarn]

Then it's as simple as extending these macros. I can do that.

[barosl]

It seems that this issue can be closed now.

5u32.to_string().capacity() -> 1
"foo".to_string().capacity() -> 4
"hello".to_string().capacity() -> 8

Now the buffer size is incremented to fit the minimum 2^n.

[thestinger]

That makes it seem like it's just pushing them without using with_capacity at all...

[barosl]

@thestinger Uhm, yeah, you're right. Now I'm worried that not using with_capacity() could impact the performance...

[thestinger]

Rust was just picking 128 as an arbitrary amount to reserve up front before. Not using a size hint will have a severe impact on performance, but clamping to 128 was worse.