Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Thermistor readouts are very noisy. #20

Open
see3d opened this issue Jul 14, 2020 · 12 comments
Open

Thermistor readouts are very noisy. #20

see3d opened this issue Jul 14, 2020 · 12 comments

Comments

@see3d
Copy link

see3d commented Jul 14, 2020

I noted that on stock FW, the thermistor readouts on my Repetier-Host Mac graph (USB connection) were very noisy. When I switched to Marlin4MPMD FW, the graphs looked rock solid. Now switching to Marlin 1.1.x, the graphs are once again noisy. The LCD readout is also showing the noise. I recall a conversation I had with Mickey that he worked some magic to get rid of the noise. I also went to a lot of effort to construct a thermistor table with him to match the readouts of the stock FW values. Did you use that table? Are you taking full advantage of the Marlin4MPMD work?

Screen Shot 2020-07-13 at 9 35 37 PM

@aegean-odyssey
Copy link
Owner

aegean-odyssey commented Jul 14, 2020

Not surprising. The Marlin4MPMD firmware effectively averages 256 adc samples per reading whereas the mpmd_marlin_1.1.x firmware averages 16 samples. This difference likely accounts for much of the "noise" that shows in your graph. And contributing to a lesser extent is that mpmd_marlin_1.1.x uses an 8-bit a 10-bit adc sample while Marlin4MPMD employs a 12-bit adc sample. The 8-bit 10-bit sample in the mpmd_marlin_1.1.x firmware is a "throw-back" to Marlin's AVR cpu origins, that is, the AVR's 8-bit 10-bit a/d converter, and my goal to change the Marlin code as little as possible.

I expect to get around to implementing the 12-bit adc sample at some point, but where I can I want my changes to align with the upstream work on Marlin 2.0 rather than Marlin4MPMD. I also have the problem of "shoe-horning" changes into very little remaining flash rom space.

As far as the extra filtering, the Marlin4MPMD technique uses precious more code space and execution time (execution time is not really at issue here, though). A better approach in this case, I think, would be to increase Marlin's "oversampling" value from 16x to perhaps 128x to achieve a similar effect. Such a change should require no extra code and does not alter the execution time of Marlin's main loop.

Also, I doubt if the "noise" in this case compromises the actual temperature control. But I think it may have some undesirable effects on Marlin's "thermal run-away" logic -- something to look into.

Anyway, I'll put this on the "to-do" list for the next firmware release. I think it's doable.

Thank you for putting the firmware through its paces. It will be great to squeeze every bit of performance and capability out of the Monoprice MP Mini Delta printer.

@aegean-odyssey
Copy link
Owner

aegean-odyssey commented Jul 31, 2020

With release 119r12, the firmware filters a/d readings with a 16x IIR (infinite impulse response) filter. Calculation-wise, the filter changes one addition to three additions and a shift; no real performance or space penalty. The filter should noticeably reduce the noise in the temperature readings.

@see3d
Copy link
Author

see3d commented Jul 31, 2020

The 119r12 changes make a notable difference. The noise is now typically below 1C. Good fix.
Thank you.

Screen Shot 2020-07-31 at 3 04 38 PM

@see3d see3d closed this as completed Jul 31, 2020
@see3d
Copy link
Author

see3d commented Aug 5, 2020

Something is not making sense to me. Running 119r13 now. I noticed that the IIR filters do not seem to be in the loop for the temperature readouts some of the time. The display looks like the screenshot below. Then after running some bed probing patterns for a while, I noticed that the readout now looks like the screenshot above with
the IIR filter.

Is the IIR filter always in the loop?

Screen Shot 2020-08-05 at 3 19 49 PM

@see3d see3d reopened this Aug 5, 2020
@aegean-odyssey
Copy link
Owner

It sure does look like no filtering is happening. But I don't think that's the case.

Marlin samples and filters the two A/D channels at about 240Hz right from startup. It transfers filtered values to "current" readings at a 15Hz rate (due to the original 16x summing filter design). From there, the "current" readings are converted to °C and stored in yet another location. As convoluted as the Marlin implementation is, it does look to be working (in the code, that is).

Are you graphing the output by polling the M105 code, or using auto report temperatures, the M155 code? I'll check on how Marlin reports the temperatures, just in case.

At 25°C, the approximate 8° variation in the chart is around 10 A/D steps. With the fairly heavy IIR filtering, though, the signal would have to be pretty noisy to drive 10 step excursions -- something like bogus readings due to not accessing the CPU's A/D properly.

Do you think the (33Hz PWM) fan could be an issue? It runs independently in hardware with very low compute or space demands. If it affects the A/D reading, the effect could look like noise. Marlin's software PWM fan (a la Marlin4MPMD) runs in same interrupt loop as the temperature readings -- in some sense the two are synchronized, so if the (soft) PWM fan were to interfere with the A/D, the effect may not appear so random.

Anyway, I'll look into the trouble.

@see3d
Copy link
Author

see3d commented Aug 6, 2020

I am seeing these errors from sending M105 at a 1 Hz rate over USB. The PWM fan was not running. Also, most of the time the reading look fine, it is only in on occasions that the noise. I have not figured out what conditions cause the noise to appear. It may be due to some state the printer is put into during different operations, or it may be completely random form some beat frequency of states coming into alignment. All I can do is stay aware of it and see if some pattern of use becomes obvious. Perhaps if there were two asynchronous processes that occasionally come into alignment and cancel or skip the IIR filtering.

@see3d
Copy link
Author

see3d commented Aug 6, 2020

I had walked away from my printer for a while. It has been sitting idle for many hours with the power on and monitoring the temps as its only activity. I just noticed that on the display it was just transitioning from the filtered to the noisy condition. Once in the noisy state it seems stuck there. Disconnecting the montoring program and reconnecting did not fix it. However, doing a G28 on the printer did alter it to a less noisy state (2nd screenshot mid screen is G28).

Screen Shot 2020-08-06 at 9 33 46 AM

Screen Shot 2020-08-06 at 9 40 48 AM

@aegean-odyssey
Copy link
Owner

Hmm, G28 will enable the stepper motors again if they've been idled (disabled) by Marlin. Marlin does employ some kind of idle timeout to turn off the stepper motors. Maybe turning off the stepper motors has something to do with this mystery.

I go back and forth on the idea that it's a mechanical problem (e.g. faulty connection). The graphs kind of have that "something is floating (not connected) look", but both the bed and hotend thermistors exhibit the trouble. I'm not sure how that could be, so a mechanical issue seems less likely to me.

It brings me back to what's in common, the A/D converter. I think garbage reads from the A/D converter after being heavily filtered could look like the graphs. I'm suspicious because the CPU manufacturer provides some errata concerning the A/D converter. I'll try to find a way to reproduce the problem on demand -- that will go a long way to tracking it down.

@see3d
Copy link
Author

see3d commented Aug 8, 2020

I might agree with the motor stop theory except I can turn the motors on and off and it does not bring the noise back. If it was something like that, then I would not expect the temps to be correct which they are. It might be that the the A/D setting gets changed to something else, like sample rate or number of bits in the sample. Right now motors on or off are both showing a well filtered output. I bet this will be a head slapper when you figure it out.

@aegean-odyssey
Copy link
Owner

Thanks for reporting. I can cross stepper motor off the list of possibilities.

The A/D converter has a calibration process that is normal run when it is first enabled. But according to the errata, there are cases when the "busy" flag for the A/D is incorrect or may get stuck. I follow the recommended work-around, and also re-initialize if something seems amiss -- which in turn, runs the calibration procedure. It's not complicated code or anything, but if it is re-initializing from time to time... well, I'm not sure what the effects would be.

Since the sampling rate is so low in this application, I'm thinking that it might be better to initialize and calibrate for every reading. Of course, one never knows, it could make things worse.

Anyway, first things first. I'll try to reliably reproduce the problem.
Thank you for your help.

@see3d
Copy link
Author

see3d commented Aug 8, 2020

BTW, I don't think I have seen this noise when the printer is printing at temperature. I might just be less aware of it because the full scale is a lot larger. However the noise is the same for both hotend and bed, so it would be in some common circuit for both perhaps. I should ask, have you seen this in your machine? We should make sure it is not something unique to my machine setup.

@aegean-odyssey
Copy link
Owner

It turns out that the transfer curve used to convert from A/D steps to °C starts to get steep at temperatures < 25°C (and > 260°C). So a few steps of the A/D translates into larger temperature swings as the temperature moves into those regions. I do see the the temperature bouncing around a bit at the low end, on the display -- but I think the display does its own thing with the temperatures as well. I need to look over some recorded temperature data before I can make any real claims about what's happening on the machine here.

The common element with the bed and hotend thermistors is the ground connection. Something going on the ground side of the A/D (noise, bad connection, ...) would show up on both channels. Also something going on with the voltage reference for the A/D would show up on both channels. And I suppose if the "official" ground connection were missing, things could still appear to work via some parasitic ground route which could certainly be noisy. Possible, but grasping at straws -- I think the software is a better suspect.

Still, another aspect of the G28 command "fixing" things is that the hotend moved. So for the sake of completeness, if you see the issue again, wiggle the hotend thermistor wire. If the issue disappears (I don't think it will) we might be on to something.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants