Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assertion Failure in Sam2p at rule.cpp:514 #77

Closed
Ngiong opened this issue Oct 22, 2020 · 0 comments
Closed

Assertion Failure in Sam2p at rule.cpp:514 #77

Ngiong opened this issue Oct 22, 2020 · 0 comments

Comments

@Ngiong
Copy link

Ngiong commented Oct 22, 2020

Hello,
We are currently working on a new fuzz testing feature, and we found a crash in sam2p.

Step to Reproduce
We used CXXFLAGS="-g -O0" ./configure to configure sam2p, and build using make, and run it with:

./sam2p -c:zip -pdf:0 -pdf:b2 -c:zip PNM: -pdf:0 -3 <attached_file> EPS:

Attachment: poc_0005.txt

Environment

  • OS: Ubuntu 18.04.4 LTS
  • Compiler: gcc 7.5.0
  • sam2p commit version: b953f63

Additional context
Here's the stack trace: stack_trace_0005.txt

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
51	../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1  0x00007ffff769b8b1 in __GI_abort () at abort.c:79
#2  0x00007ffff768b42a in __assert_fail_base (fmt=0x7ffff7812a38 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x5555555b3c68 "cacheHints.PredictorBPC!=0 && \"doSampleFormat already called\"", file=file@entry=0x5555555b3c5d "rule.cpp", line=line@entry=514, function=function@entry=0x5555555b4740 <Rule::OutputRule::appendDecoderSpec(GenBuffer::Writable&) const::__PRETTY_FUNCTION__> "void Rule::OutputRule::appendDecoderSpec(GenBuffer::Writable&) const") at assert.c:92
#3  0x00007ffff768b4a2 in __GI___assert_fail (assertion=0x5555555b3c68 "cacheHints.PredictorBPC!=0 && \"doSampleFormat already called\"", file=0x5555555b3c5d "rule.cpp", line=514, function=0x5555555b4740 <Rule::OutputRule::appendDecoderSpec(GenBuffer::Writable&) const::__PRETTY_FUNCTION__> "void Rule::OutputRule::appendDecoderSpec(GenBuffer::Writable&) const") at assert.c:101
#4  0x000055555557a9d8 in Rule::OutputRule::appendDecoderSpec (this=0x5555557fd820, out=...) at rule.cpp:514
#5  0x000055555557c1fa in Rule::writeTTE (out=..., outpal=..., outstream=..., template_=0x5555557ea058 " getinterval\n/F T`F def\n/m 0 def\n{255 div m 1 add dup 3 eq{pop setrgbcolor\n`w `h false[1 0 0 -1 0 `h]F imagemask\n0}if/m exch def}forall\n`t}\n%%BeginData:;\nexec\n`S\n%%EndData\nend restore showpage\n%%Trail"..., or_=0x5555557fd820, sf=0x7fffffffdcc0, stream_writer=0x55555557b7d6 <Rule::writePalData(GenBuffer::Writable&, GenBuffer::Writable&, Image::SampledInfo*)>, strings=0x7fffffffdac0) at rule.cpp:888
#6  0x000055555557d864 in Rule::writeTTT (out=..., outpal=..., outstream=..., template_key=0x5555555af520 "l23tran2", or_=0x5555557fd820, sf=0x7fffffffdcc0, stream_writer=0x55555557b7d6 <Rule::writePalData(GenBuffer::Writable&, GenBuffer::Writable&, Image::SampledInfo*)>, strings=0x7fffffffdac0) at rule.cpp:1093
#7  0x000055555556161a in out_l23_work (out=..., or_=0x5555557fd820, sf=0x7fffffffdcc0) at appliers.cpp:326
#8  0x000055555557b52d in Rule::applyProfile (out=..., rule_list=0x5555557fd520, sf=0x7fffffffdcc0) at rule.cpp:680
#9  0x000055555555fe93 in run_sam2p_engine (sout=..., serr=..., argv1=0x7fffffffdeb0, helpp=false) at sam2p_main.cpp:1079
#10 0x0000555555560512 in main (argv=0x7fffffffdea8) at sam2p_main.cpp:1148

Thank you.

@Ngiong Ngiong closed this as completed Nov 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant