forked from qooroo/HdrHistogram.NET
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Notes and Format.txt
100 lines (78 loc) · 4.02 KB
/
Notes and Format.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
Header cookie = 478450451
-->wordsize == 9
at
var filledLength = histogram.FillCountsFromBuffer(
buffer,
expectedCapacity,
wordsize);
The buffer has been read to the end. :(
It appears I have a mixed message now. I need to document the whole process
From LogReader->?->?
HistogramLogReader.ctor
histogramLogReader.nextIntervalHistogram
ReadsLines from file
Loads header info
Loads line prefix date
Reads line remainer from Base64 into byte array->byte buffer
EncodableHistogram.decodeFromCompressedByteBuffer(buffer, 0);
peeks the cookie value
if IsDoubleHisto : DoubleHistogram.decodeFromCompressedByteBuffer ? Histogram.decodeFromCompressedByteBuffer(
Histogram.decodeFromCompressedByteBuffer
decodeFromCompressedByteBuffer(buffer, Histogram.class, minBarForHighestTrackableValue);
Calculates HeaderSize from cookie
Gets lengthOfCompressedContents
Decompresses remaining buffer into headerBuffer
final Inflater decompressor = new Inflater();
byte[] compressedContents = new byte[lengthOfCompressedContents];
buffer.get(compressedContents);
decompressor.setInput(compressedContents);
final ByteBuffer headerBuffer = ByteBuffer.allocate(headerSize).order(BIG_ENDIAN);
decompressor.inflate(headerBuffer.array());
T histogram = decodeFromByteBuffer(headerBuffer, histogramClass, minBarForHighestTrackableValue, decompressor);
decodeFromByteBuffer(headerBuffer, histogramClass, minBarForHighestTrackableValue, decompressor);
read cookie
conditionally loads meta data (payloadLengthInBytes, normalizingIndexOffset, numberOfSignificantValueDigits, lowestTrackableUnitValue, highestTrackableValue, integerToDoubleValueConversionRatio) fro either v0 or V1+ formats
histogramClass.getConstructor(constructorArgsTypes)
.newInstance(lowestTrackableUnitValue, highestTrackableValue, numberOfSignificantValueDigits);
histogram.setIntegerToDoubleValueConversionRatio(integerToDoubleValueConversionRatio);
histogram.setNormalizingIndexOffset(normalizingIndexOffset);
calculate expectedCapacity
If !Compressed,
keep reading from the input byteBuffer
else
//Decompress the remaining data in the decompressor
ByteBuffer payLoadSourceBuffer = ByteBuffer.allocate(expectedCapacity).order(BIG_ENDIAN);
int decompressedByteCount = decompressor.inflate(payLoadSourceBuffer.array());
int filledLength = ((AbstractHistogram) histogram).fillCountsArrayFromSourceBuffer(
payLoadSourceBuffer,
expectedCapacity,
getWordSizeInBytesFromCookie(cookie));
histogram.establishInternalTackingValues(filledLength);
return histogram
===================================================================================================
V2 Format
---------------------------------------------------------------------------------------------------
#[StartTime|BaseTime|Comment] --Header and comments
[ColumnHeaders] --Human readable column headers. ignored
[LogTimeStamp][IntervalLengthSeconds][MaxTime][CompressedPayload]
[StartTime]
[StartTime:\d*.\d --start time represented as seconds since epoch. Stored as double
[BaseTime]
[BaseTime:\d*.\d --base time represented as seconds since epoch. Stored as double
[Comment]
any text --Value is ignored
[ColumnHeaders]
StartTimestamp.*
[LogTimeStamp] double
[IntervalLengthSeconds] double
[MaxTime] double. Ignored
[CompressedPayload] Base64 encoded.
Base64
Compressed
[cookie][lengthOfCompressedContents][CompressedHeader]
[CompressedHeader]
Compressed
[cookie][payloadLengthInBytes][normalizingIndexOffset][numberOfSignificantValueDigits][lowestTrackableUnitValue][highestTrackableValue][integerToDoubleValueConversionRatio][CompressedContents]
[CompressedContents]
ZigZaggedEncoded -- long encoded in a ZigZag LEB128 format
[count]* -- Read count values (as longs) and assign to current index.