-
Notifications
You must be signed in to change notification settings - Fork 831
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Decoding misses #1042
Comments
Have you checked The library's method of capturing data is by no means perfect. It can and frequently does swallow more data (messages) at one go and things it's a single message. What I suspect is happening is your buffer size (100) is pretty small, and it's seeing the start of a NEC message, and N repeats (or part there of) until it hits the buffer full ( Possible solutions/ways to address this:
Don't:
Do:
Basically. Get used to some disappointment with IR, It works most of the time, just never ALL of the time. :-/ P.S. Code style tip/wtf: |
I don't think there's a buffer overflow, as the code calls
I have to partially disagree there. The minimum time between IR bursts is ~100ms, my finger can stop holding a button and push a different one in around that amount of time with little effort.
Alas I can't. I want to intercept IR messages from my TV remote and the TV (an LG smart TV), and changing the remote (or the TV) is not an option.
I think that's the real issue. I just noticed that the IR input is indeed noisy, with glitches here and there, and that the decoding failures almost always happen when a glitch occurred slightly before the NEC header. Here's an example: For the short term, I can detect decoding failures and discard any repeat codes that come after those. From the user perspective the volume will stop changing with no apparent reason and they have to release and push the button again, but it's better than seeing the opposite reaction (volume going up while you wanted to go down... panic!) As a solution, I was wondering how feasible it is for the library to retry decoding while discarding one input transition at a time, until either the decoding succeeds or there are no more transitions. This would give a pretty good protection from early glitches. Of course the processing time may become large, but it can be contained if only one protocol has to be supported.
It's a coding style based on guard clauses (see also SO). I use it very often in functions to get all "irrelevant" cases out of the way, by exiting the function earlier, and go to the most important part of the function without unneeded indentation. |
See my comment on a dedicated state machine(s). It reasonably can't be done for all protocols. I already have stuff in each decoder to short-circuit exit as quick as possible to reduce wasted CPU time in decoding. Doing the approach you suggest pushes the problem towards a O(N^2) or O(NlogN) problem. Hence it (full state machine) can realistically only be done for a handful of protocols at the speed of the ESP chips, and would be hella-complex to maintain/debug. Doing an exhaustive search for a message somewhere in the capture buffer is something I have thought of/toyed with doing at a future date but it would require a huge undertaking and re-write, & come with a lot of limitations. e.g. Some protocols are would totally appear valid inside other valid but different protocols. Hardware vendors/manufactures/designers have the luxury of saying "I am only expecting a NEC message, everything else is invalid." This library doesn't, as it is designed to be the proverbial swiss-army knife etc.
What you really need to do is implement the control logic of the NEC protocol/device in your project. This library doesn't do any of that. It's at a low(er) level. You need to implement the control logic yourself. |
Nitpick/Off topic etc:
Oh, I'm not against guard clauses. The library is littered with them. Anyway .. it was way to early in the morning when I replied. Yes, using |
FYI, reducing the 'timeout' value will also the window for noise to be captured and included with a valid message too. (thus obscuring it) |
I'd like to go forward with the exhaustive search, and I see your concerns. How feasible would it be to separate the
I considered that but I cannot go lower than 4-5 ms anyway, or it would timeout on the NEC header. So it wouldn't eliminate other "very near" glitches. |
Of course this means also keeping the original function for compatibility. It just runs the two parts one after the other. |
Agreed. Actually 9-10ms would be the lower boundary, as the header is 8.96ms. It would be about 1/3 less chance of noise, which is still a cheap & free reduction.
I had a brief time to think about it while driving today.
This would not break backward compatibility, and use the same functions/code with easy/little modification. Thus not adding much complexity, replication of code, and keep the overall code foot print of the library pretty low. This would leave it a run-time option a user can choose. A small max skip value (3-5 pairs) would cover probably three standard deviations of noise cases anyway. It would also cap the extra cpu time to about 3-5x (near linear-ish) which is probably acceptable for your case, especially if you are reducing the number of protocols that will attempt to decode anyways. |
i.e. Not a real/proper dynamic state machine implementation, but a quick and dirty hack that will probably work, and actually, isn't that dirty the more I think about it. ;-) |
I could potentially add something that optionally tries to remove mark/spaces that are likely too short to be anything real e.g. <= ~250usecs. I think the best approach would be to have this applied to the entire capture buffer prior to decode (detecting protocols), or not at all. Doing it in each protocol decode function would be a lot of duplicate code and overhead. |
* All decoding protocols updated to use a starting offset when decoding. * `irrecv::decode()` now has two additional optional arguments. - `max_skip`: Skip over entries at the start of a capture to aggressively look for protocols to decode. Warning: Very CPU expensive! - `noise_filter`: Try to remove entries from the raw data that are smaller than this value. Danger: This will cook the raw data and potentially break some protocol decoders. * Unit tests updated to use starting offset. * New unit tests to confirm the new options work as expected. Fixes #1042
* All decoding protocols updated to use a starting offset when decoding. * `irrecv::decode()` now has two additional optional arguments. - `max_skip`: Skip over entries at the start of a capture to aggressively look for protocols to decode. Warning: Very CPU expensive! - `noise_filter`: Try to remove entries from the raw data that are smaller than this value. Danger: This will cook the raw data and potentially break some protocol decoders. * Unit tests updated to use starting offset. * New unit tests to confirm the new options work as expected. Fixes #1042
@egueli Can you please download & test PR #1046 / https://github.com/crankyoldgit/IRremoteESP8266/tree/aggressive_decoding and let me know if it breaks anything, works as expected, and fingers crossed, solves your issues. I'd love it if you can actually profile/gather timings on the ESP for how slow it gets per P.S. You owe me a Sunday! ;-) |
Thank you so much! I'll try it and do some tests right away. P.S. are you in Australia? Because my Sunday just started ;) |
Yes. Hi from the future! |
I found something: with the introduction of variable offset, the NEC protocol (and likely others) needed some more tweaking due to the fact that
While I was there I tried to make decoding tolerant to "forward" glitches, at least for the repeat messages:
Now we much nicer numbers: Now I'll have a look at noise_floor. |
Thanks for the feedback and data. I am more confident about the Looking at that image, I would have expected So, lets try capturing/displaying the raw data when you detect it as e.g. (Untested handwritten 4am code ;-)
|
Not sure you got my message at #1042 (comment) |
Yeah, I found lots of issues with length check short circuits when I was writing that PR. I'm not surprised I missed it. However, I think I'm going to have to re-work that (length check) entirely as the code you've suggested will rarely fail for a "normal" NEC message (i.e. non-repeat) |
I'd still like the data for the |
Man! 20 pairs of glitches in a signal & you still got an error rate of 2.3%! What, do you have the IR sensor in direct sunlight or something?!?! ;-) More seriously, maybe you need a better IR demodulator module. I don't think I've ever seen IR noise that bad in all my personal testing, unless I was doing something silly. Have you tried seeing how much noise you get with the sensor covered up completely? i.e. Could the source of the noise be something else? |
Here's some data from the
Here in NL we have strong wind & rain all day, and won't stop until Wednesday :/ Funnily enough, if I turn on the bright neon lamp above my desk, the error rate drops to almost zero...
I'll try that, and with another TSOP1738. |
First off, see the comments in the (new) code. Don't use For some of them (short lengths with no real long signals: e.g. However, just by visual inspection, there are plenty of real non-repeat NEC messages being marked as UNKNOWNs. I brief look at the values indicates they may be just beyond the edge of the default tolerances. I suggest you increase the tolerance values. In short, I think you should only really need to set a skip value, and not the noise filter (especially for debugging info collection) and slightly tweak the tolerance values. If you can re-do/re-collect the data without the noise filter active (i.e. set it to Thanks in advance. |
Okay, that's bizarre. I've heard that fluorescent lights are notorious for generating IR interference, but I've never heard of turning on a lamp giving noticeably less interference. That's counter-intuitive, unless there is some AGC in the module or something. i.e. so much background IR etc that it raises it's internal threshold or something. Shrug. I'm no electrical engineer. :-/ |
Ooooh! NL! Please consider adding Dutch to the i18n/locale support for this library. :-) |
Actually, look at some of the data. e.g.
The issue is the trailing noise. i.e. |
You're totally right. Below is some output with tolerance set to 30% and no noise_floor (still skip=5).
I'm Italian living in NL, I can give it a try for it-IT :)
|
Thanks for that. I'll try to incorporate some improvements based on that shortly (now I'm actually awake ;) The short UNKNOWNs of length 5-7 and no There are a good percentage of that the noise filter would/should recover. Good to know! However there are some with trailing noise, that I could handle better. I'll look into that shortly. I know we are only looking at mainly False Negatives at with this data so it's skewed. But wow, you have a noisy IR environment. You really shouldn't need this much signal processing. I don't think we are going to get to perfect in your setup. |
* Use a `uint16_t` for the noise floor to allow values large than 255. * Address an off-by-one issue which allows removal of noise at the end of a sample. * Add safe-guards in case we are near the end of the capture buffer. * Tweak NEC repeat detection to make it more strict. i.e. Require a trailing gap. - @egueli may not like this. ;-) * Add more Unit Tests based on real-world data/conditions. For #1042
@egueli Care to download the branch again and try it out, please? It should match a bit better than before. Don't expect a lot. In some cases, the data you collected is just really way too out of spec. |
Consider that the IR signal comes from another ESP8266 running this same library. Its wifi is off, but timings may still be inaccurate. Will test right away, also with my real IR remote. |
Not sure what you did exactly but wow... now it's so much better! I'm down to 0.8% failure rate (27 UNKNOWNs out of 3393 decoded), and I could see almost all of them are due to just noise. I tried with my real remote, and the "lab" results reflect the real world - decoding is much improved. In over a minute of press/hold the volume buttons, I only had one meaningful UNKNOWN for a full message (I won't report the data because noise_floor = 300) but I can live with that: in my program I can use the raw data length to recognize a full message and if so, stop repeating. I cannot thank you enough for the amount of time you spent into solving this particular issue. I hope this will benefit other users in the future! |
Yep. Welcome to IR. Sometimes the IR gods dislike us. It could be the ESP, it could be the IR demodulator, it could be coinciding with IR noise in your env. I did say earlier: "Get used to some disappointment with IR, It works most of the time, just never ALL of the time. :-/" |
A lot of that recent change was fixing the "trailing noise" problem.
So do I, I'm sure someone else out there will use these features. If you do want to thank the project, seriously, look into adding |
* Add options to `decode()` to aid detection. * All decoding protocols updated to use a starting offset when decoding. * `irrecv::decode()` now has two additional optional arguments. - `max_skip`: Skip over entries at the start of a capture to aggressively look for protocols to decode. Warning: Very CPU expensive! - `noise_filter`: Try to remove entries from the raw data that are smaller than this value. Danger: This will cook the raw data and potentially break some protocol decoders. * Unit tests updated to use starting offset. * New unit tests to confirm the new options work as expected. * Improve detection of NEC repeat codes. - Tweak NEC repeat detection to make it more strict. i.e. Require a trailing gap. * Add more Unit Tests based on real-world data/conditions. Fixes #1042 Ref: #1042 (comment) Co-Authored-By: Enrico Gueli <enrico.gueli@gmail.com>
_v2.7.4 (20200226)_ **[Bug Fixes]** - IRMQTTServer: Fix bug when receiving an IR A/C message and not re-transmitting it. (#1035, #1038) - Coolix: `setRaw()` doesn't update power state. (#1040, #1041) **[Features]** - Electra: Add improved feature support. (#1033, #1051) - Add support for Epson protocol. (#1034, #1050) - Add options to `decode()` to aid detection. Improve NEC detection. (#1042, #1046) - SamsungAc: Add support for Light & Ion (VirusDoctor). (#1045, #1048, #1049) - Add Italian (it-IT) locale/language support. (#1047) (kudos @egueli) - gc_decode: Add repeat support for pronto codes. (#1034, #1043) **[Misc]** - Update supported SamsungAc devices (#1045) - Coolix: Subtle protocol timing adjustments (#1036, #1037) - Add supported Electra device model info (#1033)
_v2.7.4 (20200226)_ **[Bug Fixes]** - IRMQTTServer: Fix bug when receiving an IR A/C message and not re-transmitting it. (#1035, #1038) - Coolix: `setRaw()` doesn't update power state. (#1040, #1041) **[Features]** - Electra: Add improved feature support. (#1033, #1051) - Add support for Epson protocol. (#1034, #1050) - Add options to `decode()` to aid detection. Improve NEC detection. (#1042, #1046) - SamsungAc: Add support for Light & Ion (VirusDoctor). (#1045, #1048, #1049) - Add Italian (it-IT) locale/language support. (#1047) (kudos @egueli) - gc_decode: Add repeat support for pronto codes. (#1034, #1043) **[Misc]** - Update supported SamsungAc devices (#1045) - Coolix: Subtle protocol timing adjustments (#1036, #1037) - Add supported Electra device model info (#1033)
The changes mentioned above have been included in the latest IRremoteESP8266 v2.7.4 release. |
Thank you! Now that it's in the current release, I could publish my project to https://github.com/egueli/SamsungM7_IR . |
Fyi: "A level shifter (because the TSOP works at 5V but the ESP) like Adafruit's" in you README is incorrect. The esp8266 handles 5V digital input. The ESP32 doesn't. |
Version/revision of the library used
v.2.7.3
Expected behavior
Actual behavior
Steps to reproduce the behavior
Circuit diagram and hardware used (if applicable)
Circuit is as described above. Using TSOP1738 and WeMos D1 mini clone.
I have followed the steps in the Troubleshooting Guide & read the FAQ
Yes.
Has this library/code previously worked as expected for you?
No.
Other useful information
I've captured the outputs from both sender and receiver with a logic analyzer. It captured three missed messages in 60s:
"IR" is the incoming IR messages from TSOP1738, "tx vol+/-" are the messages sent by the sender, "irLoop calls" is a pin that toggles every time
irrecv.decode
is called, "rx vol+/-" are the receiver's LED outputs. I would expect the rx channels to always match the tx channels.Here is the raw data, captured with Saleae Logic:
Bug.logicdata.zip
The text was updated successfully, but these errors were encountered: