[RFC] Remarkable support #41
Conversation
Fairly arbitrary mapping of waveform, dither etc :s
|
Looks fairly okay for a first stab ;). (And I'm hoping the ifdeffery mess wasn't too awful, especially since the recent cleanup ;)).
|
|
At a very quick glance at the driver, the actual dithering value doesn't matter much? https://github.com/reMarkable/linux/blob/zero-gravitas/include/uapi/linux/mxcfb.h (I'm not on my main dev box, and Kindle/Kobo kernels are only available as tarballs, so I can't have a quick comparative look at them, so take this with a grain of salt ^^). |
|
It's also highly unclear to me which version of the driver it's actually supposed to use, especially when looking at the commits in the more up-to-date zero-gravitas_4.9 branch, which mostly appear to target the older driver (https://github.com/reMarkable/linux/blob/lars/zero-gravitas_4.9/drivers/video/fbdev/mxc/mxc_epdc_fb.c && https://github.com/reMarkable/linux/blob/lars/zero-gravitas_4.9/include/uapi/linux/mxcfb.h). In which case, that dithering flag is actually meaningless (because it's only supported by the v2 driver), and I have no idea why the kernel actually supports the ioctl with that updated, larger struct ^^. (Although I have seen some weird hoop-jumping being done before in the name of backward compatibility, so, who knows ^^). |
|
The config appears to confirm that the v2 driver is unused, so you can pretty much forget about all that, and always set dithering & quant_bit to 0 ;). Still puzzled as to why they kept the updated struct, though. But I can roll with that, it's certainly less messy than what I've seen before, which is either a different ioctl/struct combo for each device, or a device with support for 2 or 3 different ioctls with a slightly different struct each time ;). |
|
I wouldn't mind a link to the upstream Kernel repo in the bundled mxcfb header copy, by the way ;). I probably ought to do that myself for the Kindle & Kobo ones, actually ^^. |
|
Extremely minor nitpick: I'm guessing they care about the peculiar casing of the |
|
And, obviously, because I kind of forgot to say it in my excitement: thanks for working on this ;). |
|
I'd also add a small It'll also come in handy if you want to implement fbdepth (as being able to switch to 8bpp would improve performance in KOReader, because RGB565 is the worst). |
Looks like this is not used as the kernel is using v1 EPDC driver. So the field is in the struct which is passed to the ioctl but ignored.
Based this mapping mainly on comments from the libremarkable enum, plus a little wild guesswork. ALso added a comment pointing to the source for the mxcfb header.
When used as a user-visible string
Currently there is only a single model of reMarkable so it's easy.
|
Thanks for the feedback, I've had a go at the improvements you suggested. The waveform mode mapping is still not 1:1. Here's the output from a simple fbink invocation, this is just after starting xochitl so should be set up in the default way. |
|
Also, I'm using |
|
Great, thanks :). Don't worry too much about the waveform mapping, I'll take a pass at it tomorrow if I can (I should be able to push to your branch), that'll probably be easier than a few more back & forth, especially since you don't have any other device to compare to ;). Good to know about fbdepth (and it being snappier is definitely the end goal. A quick, obvious test is panning a full-screen cover, for instance. You also get free software dithering as a bonus ;).). I'll probably also set a few flags to make that not entirely broken, to that end, you can try running the "rota" tool from the utils folder (it should be built alongside fbdepth w/ the utils target) and posting the output, that should confirm whether there's any weird crap happening on rotation like on some Kobo devices ;). |
|
I'm currently just using fbdepth to set the depth yea, I've so far left the rotation at the default. Here's the output of rota, I assume it means more to you that it does to me :) Let me know if there is any other things it would be useful to see the output of. |
|
I just added some default rotation settings to the deviceQuirks which seem to work OK. I'm not sure if I should be setting |
|
That was precisely what I was planning to do ;). |
|
Although, yeah, the comment in the enum is a bit misleading, as it's specific to the Kobo Libra. In practice, it means no rotation shenanigans, and honor bootRota as the "native" Portrait, which is exactly what's needed here, too. I'll add a bunch of comments to make that clearer ;). |
(And clang-format pass)
As per libremarkable recommendations.
Now that it's used somewhere else than only on the Libra ;).
Hopefully. The few constants inherited from the kernel are confusing. According to the libremarkable comments, I'm fairly confident on the A2 mapping here, but the GL16 one only makes sense by virtue of it usually always being A2 + 1 on all the platforms I'm familiar with... TL;DR: Here be dragons!
|
Okay, I've hopefully untangled the As a quick test, because it might come in handy for KOReader to have a definitive answer on that front, can you check if displaying an image w/ A2 (i.e., |
Thanks :)
Yea it looks like it does, drawing with A2 on a cleared screen just draws the black bits in the image. Running again with the default settings draws full details. |
|
@tcrs: Yay! That'll allow me to play with the constant names a bit in the header to make this all slightly less confusing, and this is good to go, thanks :). I'll be tagging a release soon-ish, but if that doesn't happen soon enough for the KOReader bits, don't hesitate to bump FBInk to a post-merge commit in koreader-base (i.e., the master branch is currently and will be kept in a sane state) ;). |
|
As an additional test, does something similar happens with (I'm wondering if it's actually GL16, or GC4). In either case, it's a far less interesting mode in practice (well, GL16 has its use in theory, but not in practice on a device with GL16_FAST), and it can be a bit weird, so do try after a separate screen clear (because if the fb content doesn't "change", both of 'em are optimized away, so a double-refresh with those is essentially a no-op, even if the content was actually redrawn, as long as it's identical). |
I saw a similar effect when I was first porting koreader (and just choosing the waveform modes at random). |
|
Huh. That's... weird :D. But good to know nonetheless, thanks ;). |
|
Sidebar: if you (or anyone else with shell access to a reMarkable, really) want to dig into this further, I can whip up the usual strace patch/build to decode mxcfb ioctls, and then it's just a matter of strace'ing the official app's ioctls while doing various different things to see what exactly it's using and when ;). (This is slightly more practical that what the libremarkable guys have been doing, which is an LD_PRELOAD hack, AFAICT). |
familiar... A2, at least, appears correct and usable, which is good, as it's the one most likely to be useful of those two ;).
Also like on other platforms, set the REGAL flag for REAGLD. Like on other platforms, this is probably a no-op and/or broken ;).
|
Okay, said strace patch is done ;). It should then be as easy as running (Hopefully, I didn't do anything stupid. If I did, strace will segfault on the first mxcfb ioctl ^^.). |
|
Hmm, see also ffcd983 Fair warning: I'm terrible at this stuff, and I don't have an IDA Pro license, so this is cobbled together via objdump ;). Speaking of, @tcrs, I wouldn't mind an updated copy of that (libqsgepaper.a & xochitl), as what's available in the libremarkale repo is over two years old ;). EDIT: Oh, but |
|
Here's a few captures using that strace binary & the options you said. I grepped all the mxcfb ioctls out of the full trace (there's constant trace noise from some networking stuff going on). This one is me just moving around the ui, opening settings, searching etc. This is drawing a load of random squiggles and then erasing most of them again This is rendering a single page of a pdf (xochitl really does take about 30s to do that)... |
|
Oh, cool, thanks ;). That seems to confirm what I dug out of The only question mark left being that "Highlight" mode (currently mapped to REAGL), which I'd take to mean A2? EDIT: While we're there, I wouldn't mind a test of DU, too (fair warning: if it behaves like on other devices, it may not actually end up displaying much of the image at all). |
|
Oh, err, it turns out the actual header is part of the SDK... \o/ #ifndef EPFRAMEBUFFER_H
#define EPFRAMEBUFFER_H
#include <QtCore/QElapsedTimer>
#include <QtCore/QFile>
#include <QtCore/QMutex>
#include <QtCore/QObject>
#include <QtGui/QImage>
#include <atomic>
#include <mutex>
class EPFrameBuffer : public QObject
{
Q_OBJECT
// Testing results:
// 0: ~780ms, initialize to white
// 1: ~170ms, mono, no flashing
// 2: ~460ms, 15 grayscale, flashing all pixels white -> black pixels
// 3: ~460ms, 15, grayscale, flashing all pixels white -> black pixels
// 4: ~460ms, 4 grayscale, flashing all pixels white -> black pixels
// 5: ~460ms, 4 grayscale, flashing all pixels white -> black pixels
// 6: ~135ms, no flashing
// 7: ~300ms, white, 2 gray, 1 black, no flashing, stuck black?
// 8: ~435ms, fast flashing, all gray, highlight, several grays, lowest color - 1 gets stuck
// 9: ~2365ms, lots of flashing, delta something?
enum Waveform {
/** Display initialization
* All -> white
* 2000ms
* ~780ms stall in driver
* Supposedly low ghosting, high ghosting in practice.
*/
INIT = 0,
/**
* Monochrome menu, text input, pen input
* All -> black/white
* 260ms to complete
* 170ms stall in driver
* Low ghosting
*/
DU = 1,
/** High quality images
* All -> all
* 480ms to complete
* 460ms stall in driver
* Very low ghosting
*/
GC16 = 2,
/** Text with white background
* 16 -> 16
* 480ms to complete
* 460ms stall in driver
* Medium ghosting
*
* Local update when white to white, Global update when 16 gray levels
* For drawing anti-aliased text with reduced flash. This waveform makes full screen update
* - the entire display except pixels staying in white will update as the new image is
* written.
*/
GL16 = 3,
/** Text with white background
* All -> all
* 480ms to complete
* 460ms stall in driver
* Low ghosting
*
* Global Update
* For drawing anti-aliased text with reduced flash and image artifacts (used in
* conjunction with an image preprocessing algorithm, very little ghosting). Drawing time
* is around 600ms. This waveform makes full screen update - all the pixels are updated or
* cleared. Use this for anti-aliased text.
*/
GLR16 = 4,
/** Text and graphics with white background
* All -> all
* 480ms to complete
* 460ms stall in driver
* Low ghosting
*
* Global Update
* For drawing anti-aliased text with reduced flash and image artifacts (used in
* conjunction with an image preprocessing algorithm, very little ghosting) even more
* compared to the GLR16 mode. Drawing time is around 600ms. This waveform makes full
* screen update - all the pixels are updated or cleared. Use this for anti-aliased text.
*/
GLD16 = 5,
/** Fast page flipping at reduced contrast
* [0, 29, 30, 31] -> [0, 30]
* 120ms to complete
* 135ms stall in driver
* Medium ghosting
*
* Local Update
* For drawing black and white 2-color images. This waveform supports transitions from and
* to black or white only. It cannot be used to update to any graytone other than black or
* white. Drawing time is around 120ms. This waveform makes partial screen update - it
* updates only changed pixels. Image quality is reduced in exchange for the quicker
* response time. This waveform can be used for fast updates and simple animation.
*/
A2 = 6,
/** Anti-aliased text in menus, touch and pen input
* All -> [0, 10, 20, 30]
* 290ms to complete
* 300ms stall in driver
*
* No flashing, black seems to be stuck afterwards?
*/
DU4 = 7,
/** Unknown1
* 435ms stall in driver
*
* Fast flashing, all gray, highlight?
* Multiple grays
* Next to lowest color gets stuck
*/
UNKNOWN = 8,
/** Init?
* 2365ms stall in driver
*
* Lots of flashing, seems to do some delta updates (inverting unchanged?)
*/
INIT2 = 9
};
public:
static EPFrameBuffer *instance();
/// Which waveforms we use for different kinds of updates
enum WaveformMode {
Initialize = INIT,
Mono = DU,
Grayscale = GL16,
HighQualityGrayscale = GC16,
Highlight = UNKNOWN
};
enum UpdateMode {
PartialUpdate = 0x0,
FullUpdate = 0x1
};
static QImage *framebuffer() {
return &instance()->m_fb;
}
Q_INVOKABLE static void setForceFull(bool force) { instance()->m_forceFull = force; }
static bool isForceFull() { return instance()->m_forceFull; }
int lastUpdateId() const { return m_lastUpdateId; }
void setSuspended(bool suspended) { m_suspended = suspended; }
bool isSuspended() const {
std::lock_guard<std::mutex> locker(fbMutex);
return m_suspended;
}
mutable std::mutex fbMutex;
qint64 timeSinceLastUpdate() const;
public slots:
static void clearScreen();
static void sendUpdate(QRect rect, WaveformMode waveform, UpdateMode mode, bool sync = false);
static void waitForLastUpdate();
private:
EPFrameBuffer();
QImage m_fb;
QFile m_deviceFile;
bool m_forceFull = false;
bool m_suspended = false;
int m_lastUpdateId = 0;
std::mutex m_timerLock;
QElapsedTimer m_lastUpdateTimer;
};
#endif // EPFRAMEBUFFER_H |
|
Okay, so I've updated the mappings according to that... Which means we're unfortunately back to square one as far as A2 is concerned... So, mildly intrigued how On the vague chance the REAGL ones behave like on Kindle, you may need to make 'em flashing/FULL ( |
I built fbink from master and tried So no idea what that means! |
|
Huh. For reference, if I take my usual eInk dithering torture test, except in Grayscale:
That's roughly how I expect it to get rendered with DU or A2:
Soo, I'm guessing a full recap is in order, with a test of every waveform mode, in order: I'm mainly concerned with identifying where the "actual" A2 behavior is hiding (if any). FWIW, GC16 & GL16 should render something like the first picture here (i.e., full of banding). CC @canselcik, because most of what was said in this PR might come in handy for libremarkable in one shape or form (also, more guinea pigs :D). |
|
OK, something funny is going on! I tested all the modes with I took a load of pictures, which ended up with Then I thought I'd double check and ran the tests again. Now they all produce something which looks like |
|
I've sometimes seen weird things happening w/ A2 , but not to that extent ;). That said, I'd check with the FWIW, |
|
Ah yes the Using All the other modes have terrible flickering for 10s and settle into a banded randomly inverted image like this: |
|
Whew, mystery solved ;). Many thanks for the tests! I'll tweak the mapping for this A2 trickery and update the documentation to strongly discourage anyone from trying other values, then, thanks! Because, ouch, that's some weird output there :D. |
This comment has been minimized.
This comment has been minimized.
I'm afraid there is much I don't understand about eink screens. I added a rather arbitrary mapping of the waveform modes in get_wfm_mode. I'm not sure what the dither bit should be set to in the update ioctl. Among other things...
Could you take a look and give me some advice on this? What I've got at the moment is happily displaying some text which is the most important hurdle for me at the moment (I'm porting this to use in my koreader port really).
This change is