Qml modules #13304

chubinou · 2024-06-07T15:43:47Z

This MR aims to bring the support of QML modules to meson, the goal is to
provide something similar to CMake qt_add_qml_module function provided by Qt
(see https://doc.qt.io/qt-6/qt-add-qml-module.html)

fixes: #6988 #9683

what's provided

qml module generation, qmldir and qrc are automatically generated
automatic type registration, C++ classes can be automatically registered in
the QML module
cache generation, the qmlcachegen tool can be invoked on qml files.

what's missing:

the scope of this PR is already large, I voluntary put aside some aspect of the module generation.

no qmltc integration
no plugin code generation
no linter target
no qml import scanning
no failsafe regarding which tool is available with a given version of Qt, some
tools (or options) are not available with older version of Qt,
no 'past version' support, this would require being able to tag version to
specific Qml files, which is not very practical in terms of API
cmake copies every qml in the build directory, maybe that's necessary for qml
tooling, I haven't investigated this

CMake argument	support	comment
`VERSION`	yes
`PAST_MAJOR_VERSIONS`	no	shipping different version of the same unit would probably require to be able to tag version per file
`STATIC / SHARED`	N/A	we don't generated the final target, the output of the function should be added to a lib/exe target
`PLUGIN_TARGET`	no	plugin is not generated
`OUTPUT_DIRECTORY`	yes
`RESOURCE_PREFIX`	yes
`CLASS_NAME`	no	plugin is not generated
`TYPEINFO`	yes
`IMPORTS`	yes
`OPTIONAL_IMPORTS`	yes
`DEFAULT_IMPORTS`	yes
`DEPENDENCIES`	yes
`IMPORT_PATH`	yes
`SOURCES`	yes	source can be provided for moc, sources should be added to the final target
`QML_FILES`	yes
`RESOURCES`	no	resources can be added to the final target independently, there is no point supporting it there
`OUTPUT_TARGETS`	N/A
`DESIGNER_SUPPORTED`	yes
`FOLLOW_FOREIGN_VERSIONING`	no
`NAMESPACE`	yes
`NO_PLUGIN`	no	plugin is not generated
`NO_PLUGIN_OPTIONAL`	no	plugin is not generated
`NO_CREATE_PLUGIN_TARGET`	no	plugin is not generated
`NO_GENERATE_PLUGIN_SOURCE`	no	plugin is not generated
`NO_GENERATE_QMLTYPES`	yes
`NO_GENERATE_QMLDIR`	yes
`NO_LINT`	no	qmllint not supported
`NO_CACHEGEN`	yes
`NO_RESOURCE_TARGET_PATH`
`NO_IMPORT_SCAN`	no	not supported
`ENABLE_TYPE_COMPILER`	no	qmltc is not supported
`TYPE_COMPILER_NAMESPACE`	no	qmltc is not supported
`QMLTC_EXPORT_DIRECTIVE`	no	qmltc is not supported
`QMLTC_EXPORT_FILE_NAME`	no	qmltc is not supported

chubinou · 2024-06-21T13:04:18Z

arch tests are failing because the CI doesn't provide QtQml for qt 6 which is a new requirement for the test

dcbaker

Thanks for working on this! I've taken a cursory glance, and there's a few smallish things I've noticed. I'd like to have another look later when I have more time to get into more of the details

dcbaker · 2024-06-27T17:18:27Z

mesonbuild/modules/_qt.py

+        for check in checklist:
+            if check not in choices:


I think you can simplify (and make a better error message) this to something like:

invalid = set(checklist).difference(choices) if invalid: return f"invalid selections {', '.join(sorted(invalid))}, valid elements are {', '.join(sorted(choices))}."

dcbaker · 2024-06-27T17:19:58Z

mesonbuild/modules/_qt.py

        preserve_paths: bool

    class HasToolKwArgs(kwargs.ExtractRequired):

        method: str
+        tools: T.List[str]


you can make this a little more powerful (and maybe remove an assert later on) with:
tools: T.List[Literal['moc, 'lrelease', <insert reset of valid tools>]]

dcbaker · 2024-06-27T17:21:48Z

mesonbuild/modules/_qt.py

+        else:
+            assert False, "unreachable"


I'm happy to trust the type annotations for the final release, though this might be useful for bringup

I changed this function quite a bit, I took inspiration from the one in gnome.py

dcbaker · 2024-06-27T17:22:45Z

mesonbuild/modules/_qt.py

+                    basename: str = os.path.basename(sourcepath)
+                    classname: str = basename.rsplit('.', maxsplit=1)[0]
+
+                    if not any(basename.endswith(ext) for ext in ['.qml', '.js', '.mjs']):


Suggested change

if not any(basename.endswith(ext) for ext in ['.qml', '.js', '.mjs']):

if not basename.endswith(('.qml', '.js', '.mjs')):

dcbaker · 2024-06-27T17:23:39Z

mesonbuild/modules/_qt.py

+                    if not any(basename.endswith(ext) for ext in ['.qml', '.js', '.mjs']):
+                        raise MesonException(f'unexpected file type declared in qml sources {s}')
+
+                    if len(classname) == 0 or '.' in classname or classname[0].islower():


Suggested change

if len(classname) == 0 or '.' in classname or classname[0].islower():

if not classname or '.' in classname or classname[0].islower():

dcbaker · 2024-06-27T17:24:50Z

mesonbuild/modules/_qt.py

+
+        cmd.extend(kwargs['extra_args'])
+
+        if namespace != '':


Suggested change

if namespace != '':

if namespace:

dcbaker · 2024-06-27T17:25:04Z

mesonbuild/modules/_qt.py

+            description=f'Qml type registration for {target_name}',
+        )
+
+    @FeatureNew('qt.qml_module', '1.0')


Suggested change

@FeatureNew('qt.qml_module', '1.0')

@FeatureNew('qt.qml_module', '1.6')

dcbaker · 2024-06-27T17:29:36Z

mesonbuild/modules/_qt.py

+        qrc_resouces: T.List[T.Union[FileOrString, build.CustomTarget, build.CustomTargetIndex, build.GeneratedList]] = []
+        all_qml_files: T.Sequence[T.Union[FileOrString, build.CustomTarget]] = list(kwargs['qml_sources']) + list(kwargs['qml_singletons']) + list(kwargs['qml_internals'])


I'm wondering if it would be beneficial to use the Interpreter.source_strings_to_files method here to remove the strings and just have list[File | CustomTarget | CustomTargetIndex | GeneratedList] here, and then not have to do string handling internally.

chubinou · 2024-06-28T15:01:01Z

some self comments on the PR:

I tried to follow Qt module directory structure

for the file MyClass.qml in the module Foo.Bar, this should looks like
Foo
|- Bar
|   |- qmldir
|   |- MyClass.qml
|   |- module.qmltypes

at the moment this is still a bit clumpsy:

for the tools to actually be able to use this, we should copy the .qml files to the Bar directory, but Meson doesn't really let me copy files to a subdirectory (I can probably cheat my way out of it, but choosed not to for the moment)
this is not really needed for the compilation parts, we could generate everything on a flat level, and do the file mapping in the qrc (it defines the internal paths of each resources). I thought that maybe I can make a special target to install the qml module tree for tooling purpose, maybe some custom target with an install_tag like in the gnome module, I haven't looked in depth at this.

AdelKS · 2024-07-12T21:48:32Z

Thank you for undertaking this work !

I wanted to give a try to registering QML modules but I am afraid I've hit against a regression introduced in this MR before I could get to it. Here's the reproducer:

git clone --depth=1 https://github.com/AdelKS/ZeGrapher.git
cd ZeGrapher
meson setup build
cd build
meson compile

which outputs this error

[59/203] Compiling C++ object src/libZeGrapherLib.a.p/meson-generated_moc_polynomialregression.cpp.o
FAILED: src/libZeGrapherLib.a.p/meson-generated_moc_polynomialregression.cpp.o 
c++ -Isrc/libZeGrapherLib.a.p -Isrc -I../src -Isubprojects/zecalculator/include -I../subprojects/zecalculator/include -I/usr/include/qt6/QtCore -I/usr/include/qt6 -I/usr/lib/qt6/mkspecs/linux-g++ -I/usr/include/qt6/QtGui -I/usr/include/qt6/QtWidgets -I/usr/include/qt6/QtNetwork -I/usr/include/qt6/QtSvg -I/usr/include/qt6/QtQuick -I/usr/include/qt6/QtQmlModels -I/usr/include/qt6/QtQmlIntegration -I/usr/include/qt6/QtOpenGL -I/usr/include/qt6/QtQml -I/usr/include/qt6/QtQuickWidgets -I/usr/include -fdiagnostics-color=always -DNDEBUG -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wextra -Wpedantic -std=c++20 -O3 '-D QT_NO_DEBUG_OUTPUT' '-D QT_NO_INFO_OUTPUT' '-D QT_NO_WARNING_OUTPUT' '-D QT_NO_DEBUG' -pipe -fPIC -DBOOST_ALL_NO_LIB -DQT_QUICKWIDGETS_LIB -DQT_QUICK_LIB -DQT_QMLMODELS_LIB -DQT_QMLINTEGRATION_LIB -DQT_OPENGL_LIB -DQT_QML_LIB -DQT_SVG_LIB -DQT_NETWORK_LIB -DQT_WIDGETS_LIB -DQT_GUI_LIB -DQT_CORE_LIB -MD -MQ src/libZeGrapherLib.a.p/meson-generated_moc_polynomialregression.cpp.o -MF src/libZeGrapherLib.a.p/meson-generated_moc_polynomialregression.cpp.o.d -o src/libZeGrapherLib.a.p/meson-generated_moc_polynomialregression.cpp.o -c src/libZeGrapherLib.a.p/moc_polynomialregression.cpp
cc1plus: fatal error: src/libZeGrapherLib.a.p/moc_polynomialregression.cpp: No such file or directory

it looks like there's a naming convention that isn't respected between meson-generated_moc_polynomialregression.cpp.o and moc_polynomialregression.cpp ? The relevant meson file is this one

Or maybe it's related to the fact that I currently pass all headers that need a moc pass on them to meson's qt.compile_moc() method. I wanted to ask how does that interact with the new qt.qml_module() method with its moc_headers kwarg, because it looked like ninja was already complaining that a target is generated twice if I passed a header, that I already moc'ed with qt.compile_moc() , to qt.qml_module()

chubinou · 2024-07-18T08:39:27Z

Hi, thanks for your interest

I wanted to give a try to registering QML modules but I am afraid I've hit against a regression introduced in this MR before I could get to it.

your repository doesn't use the feature from this MR, so I don't really see how this is a regression.

I tried to compile you project but I ended up with a lot of compilation issues (template issues in your math classes, compiled on ubuntu 22.04 with GCC 11.4), So I can't really help you there

Or maybe it's related to the fact that I currently pass all headers that need a moc pass on them to meson's qt.compile_moc() method. I wanted to ask how does that interact with the new qt.qml_module() method with its moc_headers kwarg, because it looked like ninja was already complaining that a target is generated twice if I passed a header, that I already moc'ed with qt.compile_moc() , to qt.qml_module()

qt.qml_module will moc the file itself as it needs to pass specific options to moc to extract type information, so qt.compile_moc becomes redundant

AdelKS · 2024-07-18T09:34:46Z

Hey, thanks for trying to look into it

template issues in your math classes, compiled on ubuntu 22.04 with GCC 11.4

Yeah I stopped trying to make "old" GCC or clang versions happy x)

your repository doesn't use the feature from this MR, so I don't really see how this is a regression.

Sorry, I haven't been clear enough: the master branch of my repo builds fine with latest release of meson 1.5.0, but not with this branch, this is before I make changes to meson.build to use qt.qml_module(). I need to try latest meson git commit to check, maybe my build failure has nothing to do with this MR.

qt.qml_module will moc the file itself as it needs to pass specific options to moc to extract type information, so qt.compile_moc becomes redundant

I understand. I have a noob suggestion on this aspect: I have this (maybe wrong ?) approach where I use a bash script to make a meson files() array of all the headers that contain the Q_OBJECT or Q_GADGET macros and qt.compile_moc() them. Is it doable or a good idea to change qt.compile_moc() in the same way so it always produces the type information qt.qml_module() needs, and make qt.qml_module() accept the output of qt.compile_moc() ? The benefit I see to this is to streamline how to deal with Qt's shenanigans: moc all the "special" headers, then add qml modules on top if/when needed. This at least also assumes that header files that don't have the QML_ELEMENT macro wouldn't make things harder for qt.qml_module()

AdelKS · 2024-07-18T10:24:32Z

I need to try latest meson git commit to check, maybe my build failure has nothing to do with this MR.

Okay I just tried on meson commit dfd22db4b, which is the latest meson commit before your MRs commits, and my repo could build, so this MR did at least introduce breaking changes, to qt.compile_moc() I assume ?

Would you be willing to install gcc-13 on your Ubuntu to confirm ?

add-apt-repository -y ppa:ubuntu-toolchain-r/test
apt install -y gcc-13
git clone --depth=1 https://github.com/AdelKS/ZeGrapher.git
cd ZeGrapher
CXX=g++13 meson setup build
cd build
meson compile

ferdnyc · 2024-08-03T01:50:18Z

mesonbuild/modules/_qt.py

-        for tool in self.tools.values():
-            if not tool.found():
+        for tool in kwargs['tools']:
+            assert tool in self.tools, 'tools must be in {moc, uic, rcc, lrelease}'


This is the third time that same set of tools (moc, uic, rcc, lrelease) is explicitly listed out in the code, hardcoded — that's a lot of redundancy and places to keep in sync if the list ever changes. Couldn't it be defined somewhere as a set variable, then used...

on line 273:

default=list(_set_of_qt_tools)

on line 274:

QtBaseModule._list_in_set_validator(_set_of_qt_tools)

and here:

assert tool in self.tools, f'tools must be in {_set_of_qt_tools}'

to keep things from potentially getting out of sync?

(Or, heck, at least here as...)

assert tool in self.tools, f'tools must be in {self.tools}'

fixed, though it doesn't seems that I can reference _set_of_qt_tools for the typechecker in HasToolKwArgs

Mm, it seems the arg is typed as a T.List[str]... perhaps that should be more generic.

Hmm. Is there a generic that covers both list and set? Neither Sequence nor Mapping will match both. (I suppose you could call HasToolKwArgs(list(_set_of_qt_tools)) but that's kind of ugh.)

patch has been moved to #13632 (see discussions in #13524).

Hmm. Is there a generic that covers both list and set

I don't know about this

`outfiles` is the output list of the individual commands Bug: mesonbuild/pull/13304#issuecomment-2226398671

`outfilelist` is the output list of the target, while `outfiles` is the output list of the individual commands Bug: mesonbuild/pull/13304#issuecomment-2226398671

chubinou · 2024-08-07T15:11:29Z

@AdelKS the issue was due to some bad replacements in ninja backend, i opened #13526 to fix the issue

`outfilelist` is the output list of the target, while `outfiles` is the output list of the individual commands Bug: mesonbuild/pull/13304#issuecomment-2226398671

in `func_install_data`, `rename` parametter is ContainerTypeInfo(list, str) in `module/python.py`, rename is `None` rename is stored in `build.Data` object under the type `T.List[str]`

chubinou · 2024-08-09T13:47:54Z

I updated the PR, it no longer requires the sources to be generated with a precise directory structure which should match meson philosophy.

I also added a 'devel' install target to allow installing the module definition (for qmllint or and qml tools)

This allows checking for tools that may not be available in older version of qt or avoiding to request tools that may not be necessary for a given project

This aims to bring the support of QML modules to meson, the goal is to provide something similar to CMake `qt_add_qml_module` function provided by Qt (see https://doc.qt.io/qt-6/qt-add-qml-module.html ) Fixes: mesonbuild#6988, mesonbuild#9683

chubinou requested a review from jpakkane as a code owner June 7, 2024 15:43

chubinou changed the title ~~Qml modules~~ Draft: Qml modules Jun 7, 2024

chubinou force-pushed the qml_modules branch from ad556f5 to e16328a Compare June 7, 2024 16:39

chubinou force-pushed the qml_modules branch 2 times, most recently from 5a309af to 2c9b9eb Compare June 21, 2024 12:01

chubinou changed the title ~~Draft: Qml modules~~ Qml modules Jun 21, 2024

dcbaker added the module:qt label Jun 27, 2024

dcbaker requested changes Jun 27, 2024

View reviewed changes

ferdnyc reviewed Aug 3, 2024

View reviewed changes

chubinou added a commit to chubinou/meson that referenced this pull request Aug 7, 2024

backend/ninja: fix bad @OUTPUTn@ replacements

6d18bbf

`outfiles` is the output list of the individual commands Bug: mesonbuild/pull/13304#issuecomment-2226398671

chubinou mentioned this pull request Aug 7, 2024

backend/ninja: fix bad @OUTPUTn@ replacements #13526

Open

chubinou mentioned this pull request Aug 9, 2024

Allow skipping the detection of specific tools in qt.has_tools() #13524

Open

interpreter: fix signatute type of install_data_impl

c20ef07

in `func_install_data`, `rename` parametter is ContainerTypeInfo(list, str) in `module/python.py`, rename is `None` rename is stored in `build.Data` object under the type `T.List[str]`

chubinou force-pushed the qml_modules branch from 2c9b9eb to ccd6fa2 Compare August 9, 2024 13:39

chubinou added 2 commits August 9, 2024 16:03

qt module: allow has_tools to specify which tools to check

b81e0fe

This allows checking for tools that may not be available in older version of qt or avoiding to request tools that may not be necessary for a given project

qt module: allow moc to generate json introspection file

e7b5adc

chubinou force-pushed the qml_modules branch from ccd6fa2 to 3e31346 Compare August 9, 2024 14:04

chubinou added 2 commits August 9, 2024 16:56

qt module: provide qml_module

b81f9df

This aims to bring the support of QML modules to meson, the goal is to provide something similar to CMake `qt_add_qml_module` function provided by Qt (see https://doc.qt.io/qt-6/qt-add-qml-module.html ) Fixes: mesonbuild#6988, mesonbuild#9683

qt module: add qml module test

e8e8465

chubinou force-pushed the qml_modules branch from 3e31346 to e8e8465 Compare August 9, 2024 14:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Qml modules #13304

Qml modules #13304

chubinou commented Jun 7, 2024

chubinou commented Jun 21, 2024

dcbaker left a comment

dcbaker Jun 27, 2024

dcbaker Jun 27, 2024

chubinou Aug 9, 2024

dcbaker Jun 27, 2024

chubinou Aug 9, 2024

dcbaker Jun 27, 2024

chubinou Aug 9, 2024

dcbaker Jun 27, 2024

chubinou Aug 9, 2024

dcbaker Jun 27, 2024

chubinou Aug 9, 2024

dcbaker Jun 27, 2024

chubinou Aug 9, 2024

dcbaker Jun 27, 2024

chubinou commented Jun 28, 2024

AdelKS commented Jul 12, 2024 •

edited

Loading

chubinou commented Jul 18, 2024

AdelKS commented Jul 18, 2024 •

edited

Loading

AdelKS commented Jul 18, 2024

ferdnyc Aug 3, 2024

chubinou Aug 9, 2024

ferdnyc Aug 12, 2024

chubinou Sep 4, 2024

chubinou commented Aug 7, 2024

chubinou commented Aug 9, 2024

	if not any(basename.endswith(ext) for ext in ['.qml', '.js', '.mjs']):
	if not basename.endswith(('.qml', '.js', '.mjs')):

	if len(classname) == 0 or '.' in classname or classname[0].islower():
	if not classname or '.' in classname or classname[0].islower():

	@FeatureNew('qt.qml_module', '1.0')
	@FeatureNew('qt.qml_module', '1.6')

		qrc_resouces: T.List[T.Union[FileOrString, build.CustomTarget, build.CustomTargetIndex, build.GeneratedList]] = []
		all_qml_files: T.Sequence[T.Union[FileOrString, build.CustomTarget]] = list(kwargs['qml_sources']) + list(kwargs['qml_singletons']) + list(kwargs['qml_internals'])

Qml modules #13304

Are you sure you want to change the base?

Qml modules #13304

Conversation

chubinou commented Jun 7, 2024

chubinou commented Jun 21, 2024

dcbaker left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chubinou commented Jun 28, 2024

AdelKS commented Jul 12, 2024 • edited Loading

chubinou commented Jul 18, 2024

AdelKS commented Jul 18, 2024 • edited Loading

AdelKS commented Jul 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chubinou commented Aug 7, 2024

chubinou commented Aug 9, 2024

AdelKS commented Jul 12, 2024 •

edited

Loading

AdelKS commented Jul 18, 2024 •

edited

Loading