-
Notifications
You must be signed in to change notification settings - Fork 756
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add W605 warning for invalid escape sequences in string literals #676
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
pycodestyle.py
Outdated
string = text[start:-len(quote)] | ||
|
||
if 'r' not in prefix: | ||
pos = 0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not do:
pos = string.find('\\')
while pos >= 0:
pos += 1
if string[pos] not in valid:
yield (...)
pos = string.find('\\', pos + 1)
Starting with Python 3.6, invalid escape sequences in string literals are now deprecated. In a future version of Python, invalid escape sequences will be a syntax error. While this deprecation produces a runtime warning, it only appears if warnings are enabled and the first time the Python source is compiled to byte code. By adding a check to pycodestyle, projects can take advantage of static analysis to catch and fix these future syntax errors. For more information on the deprecation, see the Python release notes, https://docs.python.org/3/whatsnew/3.6.html#deprecated-python-behavior > A backslash-character pair that is not a valid escape sequence now > generates a DeprecationWarning. Although this will eventually become a > SyntaxError, that will not be for several Python releases. Fixes #633
Thanks for the review. I agree, that solution is simpler. I've made the suggested change. |
jibsheet
added a commit
to openedx/xqueue
that referenced
this pull request
Apr 12, 2018
powersj
pushed a commit
to canonical/curtin
that referenced
this pull request
Apr 13, 2018
A bit of information from python doc that I got by never having known: String literals may optionally be prefixed with a letter `r' or `R'; such strings are called raw strings and use different rules for backslash escape sequences. ... Unless an `r' or `R' prefix is present, escape sequences in strings are interpreted according to rules similar to those used by Standard C. So basically, any use of \ not followed by one of [\'"abfnrtv] or \ooo (octal) \xhh (hex) or a newline is invalid. This is most commonly seen for us in regex. To solve, you either: a.) use a raw string r'...' b.) correctly escape the \ that was not intended to be interpreted. Python has deprecated these invalid string literals now https://bugs.python.org/issue27364 and pycodestyle is identifying them with a W605 warning. PyCQA/pycodestyle#676
smoser
added a commit
to canonical/cloud-init
that referenced
this pull request
Apr 18, 2018
Python has deprecated these invalid string literals now https://bugs.python.org/issue27364 and pycodestyle is identifying them with a W605 warning. PyCQA/pycodestyle#676 So basically, any use of \ not followed by one of [\'"abfnrtv] or \ooo (octal) \xhh (hex) or a newline is invalid. This is most comomnly seen for us in regex. To solve, you either: a.) use a raw string r'...' b.) correctly escape the \ that was not intended to be interpreted.
Guts
added a commit
to isogeo/isogeo-2-office
that referenced
this pull request
May 22, 2018
openstack-gerrit
pushed a commit
to openstack/trove
that referenced
this pull request
Jul 3, 2018
Starting with Python 3.6, invalid escape sequences in string literals are now deprecated[1]. This influence also automatic style checkers like pycodestyle which starting with 2.4.0 complains about invalid escape sequences (W605)[2]. Let's fix all those warnings at once by using raw strings where possible and adding additional \ where not. Footnotes: 1 - https://docs.python.org/3/whatsnew/3.6.html#deprecated-python-behavior 2 - PyCQA/pycodestyle#676 Change-Id: I009a366fd8342edfd30890df6fe8e1fca88bf3cc Signed-off-by: Krzysztof Opasiak <k.opasiak@samsung.com>
openstack-gerrit
pushed a commit
to openstack/openstack
that referenced
this pull request
Jan 8, 2019
* Update trove from branch 'master' - update spelling error Change-Id: I4d260b00220c999072e193f5289fa1212074dcf9 - Execute functional test jobs running under python3 Functional tests are executed by trovestack script, which uses python in default OS version which is 2.7. This change wraps python in tox and executes trovestack int_tests in virtual enviroment, starting them in python3. Any future python version change could be managed by tox framework. Change-Id: I3a849978241d3d0669ef9e1c802ff504ac3c32cb Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com> - Use Ubuntu Xenial distro for integration tests Default Ubuntu release which is used under devstack, has been changed from Xenial to Bionic [1]. Fortunately, backward compatibility for Xenial was added. Until we are not compatible with Bionic, we need to use older Ubuntu distro. [1] https://review.openstack.org/#/c/610977/ Change-Id: I66c63cf4fa6e5ac201a0fef1d4a8d26520b3b803 Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com> - Merge "Add missing ws separator between words" - Merge "Set Tempest's service_availability setting for Trove" - Change openstack-dev to openstack-discuss Mailinglists have been updated. Openstack-discuss replaces openstack-dev. Change-Id: I1063113e64f698ff4cb45a4dde78242a16050a37 - Merge "Update http to https" - Set Tempest's service_availability setting for Trove Tempest's service_availability config option includes all the service availability which is further used by tests to take decision of skip or run the test. For example, [service_availability].trove is true then, trove test will run or if [service_availability].trove is false then, all trove related tests either in trove tempest plugin or any other plugins will be skipped. Currently this setting for trove service[1] is in devstack lib/tempest which is being removed by - https://review.openstack.org/#/c/619973/ For better maintenance, we are moving all tempest non-owned service setting to service devstack plugin side. This commit add the setting of trove service on trove devstack plugin. Closes-Bug: #1743688 [1] https://git.openstack.org/cgit/openstack/trove-tempest-plugin/tree/trove_tempest_plugin/config.py#n17 Needed-By: https://review.openstack.org/#/c/619973/ Change-Id: Ie3c6bab6e0be9dfcfe29ec3f412e704f3ad79d61 - Add missing ws separator between words This is to add missing ws separator between words. Change-Id: I6debb62be4577a45962f206448a62a14a4199488 - [fix_typos] fix wrongly spell word "configration " Change-Id: I174ffd6e5799aa62df1d135b595c2958a902560a - Update http to https modify http link to https link Change-Id: I9550e78f3e70885b276f5623fb1cb6b32b6ce892 - Fix the wrong url Fix the wrong url of HACKING.rst in README.rst Change-Id: I4dbd1fbfe434612116ddebc02228b00b5ea4a369 - Merge "Add python 3.6 unit test job" - Extend contribution doc by IRC contact details Add details how to contact with trove members over IRC channel. Change-Id: I00ee60812c6a46f26c7eba24711941d9b5362c96 - Add python 3.6 unit test job This is a mechanically generated patch to add a unit test job running under Python 3.6 as part of the python3-first goal. See the python3-first goal document for details: https://governance.openstack.org/tc/goals/stein/python3-first.html Change-Id: I2b04e865fb72296f672ab206fcda15e30450db97 - Merge "Fix home direcroty path in trovestack script" - Merge "Fix incorrect test_group in zull job definition" - Fix incorrect test_group in zull job definition Both of mysql trove-scenario jobs performed the same tests group (mysql-supported-single). This change sets correct group for trove-scenario-mysql-multi job. Furthermore, to be consistent, trove_test_datastore has been passed explicite for trove-scenario-mysql-single job. Change-Id: I377e17ebf690c79d1c369116697177f992b3df65 Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com> - Fix home direcroty path in trovestack script This simple fix solves "no such file or directory" error when user home directory is not prefixed by /home. This is the case, for example in devstack, where stack user home is set to /opt/stack/. Lets use $HOME variable to determine user home location. Change-Id: I07432d27e1a2035296bc62793fecbe13d0265e7b Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com> - Fix the conflict of urlparse between python2 and python3 Change-Id: I47d7a524afaa3ae74b7c6c2931d8ebb89438c508 - Merge "Add detailed list for instances" - Add trove-status upgrade check command framework This adds basic framework for trove-status upgrade check commands. For now it has only "check_placeholder" check implemented. Real checks can be added to this tool in the future. Change-Id: Idfeab4c06cba6f841c17ab6e255a29e8707bfa55 Story: 2003657 Task: 26162 - Merge "Increment versioning with pbr instruction" - Fix build ubuntu-geust issue on arm64 To build ubuntu-geust on arm64, libffi-dev & libssl-dev are required Closes-Bug: #1797014 Change-Id: Ia215deac11976fc9fc67e6d823ae11d181bf8083 Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org> - Increment versioning with pbr instruction With moving away from required milestone releases, the version numbers calculated by PBR on the master branch will not work for those testing upgrades from the last stable release. More details can be found in the mailing list post here: http://lists.openstack.org/pipermail/openstack-dev/2018-October/135706.html This is an empty commit that will cause PBR to increment its calculated version to get around this. PBR will see the following which will cause it to increment the version: Sem-Ver: feature Please merge this patch as soon as possible to support those testing upgrades. Change-Id: I2520aaa18a5f7add4356374bb709513669d8d915 Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com> - Be compilance with latest oslo.messaging Support for the [oslo_messaging_rabbit] section has been removed in oslo.messaging==9.0.0 package [1]. That's why integration tests failures on build devstack step. This is the moment where we need to use the transport_url directive in the [DEFAULT] section instead. Moreover rpc_backend property (which was used in trove to enable fake RPC backend) has been removed from DEFAULT section and API tests fails on TIME_OUT as they starts on rabbit RPC backend. Fake RPC can be now configured as described here [2]: transport_url = 'fake:/' [1] https://docs.openstack.org/releasenotes/oslo.messaging/unreleased.html [2] https://docs.openstack.org/oslo.messaging/latest/configuration/conffixture.html Change-Id: Id6c5a9198d5a213cb085407a1d8b534e7c755f69 Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com> - Add detailed list for instances Currently, listing instances only allows to get basic information about entities. To get the details, one need to query instance "show" endpoint for each instance separately. This is inefficient and exposes API to a heavier load. There are use cases in which we want to obtain detailed information about all instances. In particular, in services integrating with Trove. For example, Vitrage project requires this information to build vertices and edges in the resource graph for RCA analysis. Change-Id: I33252cce41c27cc7302c860dde1f6448ecdf3991 Signed-off-by: Bartosz Zurkowski <b.zurkowski@samsung.com> - Merge "Fix cover job" - Merge "Fix the mysql start up issue after restore" - Merge "Replace 14.04 trusty with 16.04 xenial." - Merge "Update character set and coallaction for mysql5.7" - Merge "Add blueprints and bugs link in documents" - Fix the mysql start up issue after restore When the trove guest image is built without the proper permission setting for /tmp directory in which ini file is created. (could happen when using non-official ways of image building), mysqld_safe will fail to reset the root password after mysql restore due to AppArmour complaints. As a consequence it fails the whole mysql restore procedure. Changing init-file ownership fixes this problem and makes the backup and restore successful. Co-Authored-By: Jake Yip <jake.yip@unimelb.edu.au> Closes-Bug: #1791878 Change-Id: Ia164345239197f93fb63697212a9b96c900fbe89 - Fix cover job The job is currently broken, it runs with testr while we switched to stestr. Use the usual setup for coverage as done by other projects using stestr in cover. Previously, we run with concurrency=1 - continue to run in serial mode. Change-Id: I6e389db5aed8db25b49906b2479af081237fc909 - Merge "Cleanup zuul.yaml" - Add blueprints and bugs link in documents Change-Id: I0d5e1b427506e3965b892b130aca4022619de5ae - endpoint_type option not used with single tenant Currently we are not able to specify the endpoint_type for Neutron, Nova and Cinder clients with single tenant. publicURL is configured by default but it could be nice to have the possibility to choose anything else. Change-Id: Ibb791cacc0e08de2d87b4348f84c9e573849ec51 Closes-Bug: #1776229 - Merge "Add release note link in README" - Merge "make tox -e pylint only run pylint" - Cleanup zuul.yaml Small cleanups: * Use openstack-cover-jobs template to move cover job to check, remove post cover job. Mark it non-voting for now. * Use openstack-lower-constraints-jobs, remove jobs * Sort list of templates. Change-Id: I2fad568f253a01a3ca272d2e0af4f747b9682b2c - add python 3.6 unit test job This is a mechanically generated patch to add a unit test job running under Python 3.6 as part of the python3-first goal. See the python3-first goal document for details: https://governance.openstack.org/tc/goals/stein/python3-first.html Change-Id: I8b1d6f970057175487253f43fdcbbd9f969fe2a7 Story: #2002586 Task: #24342 - switch documentation job to new PTI This is a mechanically generated patch to switch the documentation jobs to use the new PTI versions of the jobs as part of the python3-first goal. See the python3-first goal document for details: https://governance.openstack.org/tc/goals/stein/python3-first.html Also, add docs tox environment so that the job works. Change-Id: I8dd911d46623c0185d238b293e0218cfa435ed46 Story: #2002586 Task: #24342 - Merge "import zuul job settings from project-config" - make tox -e pylint only run pylint Having the pylint environment run redundant commands just makes the job take longer. Change-Id: Ibb86b40e589d1b6f9c3af5901549d7aa5170e8c8 Signed-off-by: Doug Hellmann <doug@doughellmann.com> - fix tox python3 overrides We want to default to running all tox environments under python 3, so set the basepython value in each environment. We do not want to specify a minor version number, because we do not want to have to update the file every time we upgrade python. We do not want to set the override once in testenv, because that breaks the more specific versions used in default environments like py35 and py36. Change-Id: I9ee34642c700d1e6ba9c2f3891b7fa1f7f7e1e1d Depends-On: I8989fd4798e80eae27408017e1543819a68b4ab1 Signed-off-by: Doug Hellmann <doug@doughellmann.com> Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com> Co-Authored-By: Doug Hellmann <doug@doughellmann.com> - update pylint to 1.9.2 The older version of pylint being used does not work correctly under python 3. In order to be able to update the pylint job to run under python 3, we need to update the tool. This patch updates to the latest version at this time. It also updates and pins astroid, which was previously capped. Using a pin instead of a cap should let us avoid issues with new releases while being specific about which version to actually use. Disable not-callable because that appears to be a new rule that is confused by the use of properties to access things that are set to callables. Co-Authored-By: Fan Zhang <zh.f@outlook.com> Co-Authored-By: Marcin Piwowarczyk <m.piwowarczy@samsung.com> Change-Id: I65705804b222dcd30a653fe10be3d823fa6143ff Signed-off-by: Doug Hellmann <doug@doughellmann.com> - Use latests version of python-troveclient in tests This patchset changes version of python-troveclient package version used in tests. So far, version from global constrain file was used [1]. In order to achive this, a local constrain file for tests requirements has been added. This made it possible to override global constrain versions. [1] openstack/requirements/upper-constraints.txt Change-Id: Ic4383f0249e5bd367461f63f66217af959aefba4 Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com> - import zuul job settings from project-config This is a mechanically generated patch to complete step 1 of moving the zuul job settings out of project-config and into each project repository. Because there will be a separate patch on each branch, the branch specifiers for branch-specific jobs have been removed. Because this patch is generated by a script, there may be some cosmetic changes to the layout of the YAML file(s) as the contents are normalized. See the python3-first goal document for details: https://governance.openstack.org/tc/goals/stein/python3-first.html Change-Id: Ida3b1bc70fbd6ab958d8669e925de1d3151f69ff Story: #2002586 Task: #24342 - Replace 14.04 trusty with 16.04 xenial. Change-Id: If235695625813e24810144f524f0a649f180d3c4 Closes-Bug: #1780184 - Merge "Enable mutable config in trove" - Merge "Remove unused imports from the integration tests" - Enable mutable config in trove New releases of oslo.config support a 'mutable' parameter to Opts. oslo.service provides an option here Icec3e664f3fe72614e373b2938e8dee53cf8bc5e allows services to tell oslo.service they want mutate_config_files to be called by passing a parameter. This commit is to use the same. This allows trove to benefit from I1e7a69de169cc85f4c09954b2f46ce2da7106d90, where the 'debug' option (owned by oslo.log) is made mutable. we should be able to turn debug logging on and off by changing the config. tc goal: https://governance.openstack.org/tc/goals/rocky/enable-mutable-configuration.html Co-Authored-By: Anusree A <anusree.a04:@gmail.com> Change-Id: I02da8ce44d52b0f0dc67660c8ec76b5effb5f73c - Merge "Format service apache2 reload section for Debian/Ubuntu" - Remove unused imports from the integration tests Change-Id: I2295b9b9e0435bf459364c2dab7852cc9535edfe Co-authored-by: wangqi <wang.qi@99cloud.net> Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com> - Use print function rather than statement in tests Python3 does not support the print statement, only the print function. Whereas Python 2.6 and 2.7 support both. Until statement is used, executing pylint via python3 on these files causes error. There is no more old way print in trove repository. Change-Id: I063eb9dd7e690ac2cdc92810982c7f9a6ae22388 Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com> - Merge "Reject zero volume size in API" - Update reno for stable/rocky Change-Id: I03f5565b9895deac1a66e2789e163c510a9b936a - Reject zero volume size in API According the discussion[1], set volume size to 0 makes no sense. This fix reject zero value of volume size with integer or string type. The previous commit[2] has reject negative volume size, with this fix, value like 0, '0' will be rejected and value like 001, '040' will pass. [1] https://review.openstack.org/#/c/589034/ [2] https://review.openstack.org/#/c/164460/ Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com> Change-Id: If18a3c69a2815086cc60f9a6a24e6254cc7856df - Merge "Remove nova conf for guestagent" - Merge "Add extended properties support for mongo cluster." - Add extended properties support for mongo cluster. Currently when create a mongodb cluster, mongos and configsvr use the volume_size of replica-set node. But mongos and configvr are not data node, they don't need volume space as large as data node. This patch attend to help user specify the number, the volume size and the volume type of mongos/configserver with extended_properties[1] argument when creating mongodb. Currently, the supported parameters are, num_configsvr, num_mongos, configsvr_volume_size, configsvr_volume_type, mongos_volume_size and mongos_volume_type. [1] https://review.openstack.org/#/c/206931/ Closes-Bug: #1734907 Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com> Change-Id: Ie48f3961b21f926f983c6713a76b0492952cf4c7 - Remove nova conf for guestagent It seems that the current guestagent only needs to communicate through amqp, and does not need to interact with nova, so the related nova configuration can be removed from the guestagent, which can reduce some invalid configuration. Similarly, we removed the related configuration of condutor and guestagent in devstack plugin. Change-Id: I3f0b3b445c053351c1f3e966d77ea228df0e94a7 - Fix replication failure when Swift isn't available * add Swift token verification in create instance method in Trove API when slave_id is given * catch ConnectionError exception in verify_swift_auth_token * add Swift token verification to guestagent before backup and restore * add new exception representing Swift connection error * set missing fault information when replication snapshot fails * mock verify auth token method in replication and restore unit tests Closes-Bug: #1395523 Change-Id: I6a21ba2ba890a82875f9b6dae3c6b93bc9fdb4b0 Signed-off-by: Dariusz Krol <d.krol@samsung.com> - Merge "More reliable gpg keys importing in DIB elements" - Merge "Sync the data fields of DB* class and table fields." - Merge "Migrate to Zuul v3 native job definitions" - Merge "[doc] Use openstack client command to replace others" - Merge "Update auth_uri option to www_authenticate_uri" - Merge "Trivial: Update pypi url to new url" - Sync the data fields of DB* class and table fields. This update try to fix some problems: 1. define the 'id' both in _data_fields and _auto_generated_attrs. 2. some of tables don't have the 'id' field. 3. _data_fields of DBInstance miss some table fields. 4. Change the key of persisted_models as the same of table name to make it more readable. Change-Id: I0ad96824e8de978ff5b6766085549b1e1a2509cf Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com> - Migrate to Zuul v3 native job definitions This patch migrates the legacy gate jobs to Zuul v3 native ones(a new trove-devstack role is created for all those legecy jobs, and the previous way of running gate jobs is still kept as we still cannot migrate the genade job to a Zuul v3 native one), following the documents below: https://docs.openstack.org/infra/manual/zuulv3.html https://docs.openstack.org/devstack/latest/zuul_ci_jobs_migration.html Change-Id: I10a1adae3a79b7322118fce180119f0ee5ff2e1b Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - More reliable gpg keys importing in DIB elements We're still seeing apt-key failed to import gpg keys these days during the images building in the gate jobs, the problem is keys.gnupg.net and keyserver.ubuntu.com are both not stable according to [1] and [2], it's better to adopt pool.sks-keyservers.net instead and with simple retries. To reduce code duplication, this common apt-key importing function is also moved to ubuntu-guest as an environment snippet. [1] https://www.gnupg.org/faq/gnupg-faq.html#new_user_default_keyserver [2] https://sks-keyservers.net/overview-of-pools.php Closes-Bug: #1579094 Change-Id: I0fe200d140f6f9c4d423dd498797a225e3295a71 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Format service apache2 reload section for Debian/Ubuntu Change-Id: I5e0c9a8f38a64208eb885086910b60da6a6b4ab7 - Merge "py3.x: Fix usage of gettext.install" - [doc] Use openstack client command to replace others The neutron client commands will be deprecated, and 'nova net-list' has already been deprecated, so use openstack client commands to replace neutron client. Change-Id: Idb7d9782bb3af28cba7e318b0ed664641b5d731f - Merge "Raise timeout for instance resizing checking job" - Merge "Add volume_type to apischema" - Raise timeout for instance resizing checking job Recently the trove-functional-mysql usually failed by timeout at the test_instance_returns_to_active_after_resize case, and the cause is creating and resizing instances could be quit slow(e.g. 1200+ seconds). This may be caused by the pressure of the CI system and should be further investigated. The patch is a quick fix for the problem by raising the timeout to 30 minutes(it's now 900 seconds by default, but instead it's 32 minutes for instance creating, so raising the value to 30 minutes should be reasonable). Partial-Bug: #1778837 Change-Id: Ib903a72b098499553978ec881843a7bbee25b45f Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Merge "Fix invalid escape sequence warnings" - Fix invalid escape sequence warnings Starting with Python 3.6, invalid escape sequences in string literals are now deprecated[1]. This influence also automatic style checkers like pycodestyle which starting with 2.4.0 complains about invalid escape sequences (W605)[2]. Let's fix all those warnings at once by using raw strings where possible and adding additional \ where not. Footnotes: 1 - https://docs.python.org/3/whatsnew/3.6.html#deprecated-python-behavior 2 - https://github.com/PyCQA/pycodestyle/pull/676 Change-Id: I009a366fd8342edfd30890df6fe8e1fca88bf3cc Signed-off-by: Krzysztof Opasiak <k.opasiak@samsung.com> - Merge "change pylint wrapper to let messages be prefixes" - Merge "Cleanup testrepository and os-testr requirements" - Add release note link in README Change-Id: I658ade892cd08a1e5a2be8a8a2883f417032b8b9 - Merge "Update Trove's README" - py3.x: Fix usage of gettext.install This needed to be updated to get py3.x to work. Change-Id: I987854c0266566e1e7722478dbb3e27bcffbb08b Closes-Bug: #1777440 - Remove pycrypto from requirements Remove pycrypto completely from requirements and fix the related tests. Closes-Bug: #1749574 Depends-On: I5c0c1a238023c116af5a84d899e629f1c7c3513f Change-Id: Ibfedd9e2ab0a5e78959108112f57103a089f02d1 Signed-off-by: Fan Zhang <zh.f@outlook.com> - change pylint wrapper to let messages be prefixes The messages for many of the errors reported by pylint have been extended to include "suggestions" (especially in the case of undefined variables or members). Rather than update all of the individual messages in the configuration file, just treat the messages as prefixes. Change-Id: I1f11c50e39dbd43c4a9b54ccaaccf03f5ff07aca Signed-off-by: Doug Hellmann <doug@doughellmann.com> - Update Trove's README Update the README.rst, change the repo url from github to git.openstack.org, add link for trove-sepcs and trove-dashboard, add link for CONTRIBUTING.rst and HACKING.rst. Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com> Change-Id: If1c7d3467c7da261ac626325a44aa7eab37ae042 - Add volume_type to apischema Currently, trove had support pass volume_type during creation of instance/cluster, but the schema "volume" only contains the "size", it is nesscessary add the "type" to schema "volume". Besides, there is a "required: True" under "properities", it is unnecessary. Change-Id: Id7c4317b0ba62d751956ddb1baee70fe27bc15f4 Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com> - Merge "Switch to cryptography from pycrypto" - Add a hook for restore process to check if successful 1.This is the same solution with the backup process. Previously we only check the stderr of the restore command, but this method is not reliable, and the changes during percona-xtrabackup 2.4.11: https://jira.percona.com/browse/PXB-1542 causes the gate jobs failed in creating a slave MySQL instance. The new hook currently is only used by InnoBackupEx restore runner with checking the exit status and stderr output of xbstream. 2.with[1] merged,this makes DIB_CLOUD_IMAGES more flexible, but it break trovestack build image, now we need to specify a more detailed path to DIB_CLOUD_IMAGES to get the appropriate content. [1]:https://review.openstack.org/#/c/568697/ Co-Authored-By: Zhao Chao <zhaochao1984@gmail.com> Co-Authored-By: zhanggang <zhanggang@cmss.chinamobile.com> Co-Authored-By: jiansong <jian.song@easystack.cn> Closes-Bug: #1771990 Change-Id: Ibb54d6b5953a775be339fb991a0771961d27eba2 - Switch to cryptography from pycrypto PyCrypto isn't active developed for quite a while, cryptography is recommended instead. This patch does this migration, but still keeps pycrytpo as a fallback solution. Random generation is also migrated to os.urandom as the cryptography document suggests: https://cryptography.io/en/latest/random-numbers/ Closes-Bug: #1749574 Change-Id: I5c0c1a238023c116af5a84d899e629f1c7c3513f Co-Authored-By: Fan Zhang <zh.f@outlook.com> Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Update character set and coallaction for mysql5.7 Our character set and collaction are also kept in the older version (possibly mysql5.5), which makes it impossible for us to create new character set and collaction through database-create. At the same time, we can't display these manually created database. Update here. More information comes from: https://dev.mysql.com/doc/refman/5.7/en/charset-charsets.html Change-Id: I14c0392e19c7846c7aed1e60fe5b5ca01f22a2f4 - Trivial: Update pypi url to new url Pypi url changed from [1] to [2] [1] https://pypi.python.org/pypi/<package> [2] https://pypi.org/project/<package> Change-Id: I3470831e34b3bbeb2fb028b71312f51e5224b502 - Merge "Fix dict iteration in PropertiesCodec" - Cleanup testrepository and os-testr requirements We have migrated to stestr for a while, however the related requirements is not updated, this patch does the cleanup, also updates requirements for stestr. Change-Id: I9c8cf2f90ced2261293f91faefc3099740540fe7 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Run unittests under the low-constraints job The low-constraints job should run unittests by the original desgin, we didn't put the commands about running unittests inside the default testenv settings, so in fact the low-constraints job did nothing except creating a virtualenv. This patch fixes and also move the unittests commands to the default testenv settings like the other OpenStack projects. Change-Id: Ia344a14a9baf7574b043fc08332869083da6dc1e Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Fix dict iteration in PropertiesCodec We're updating a dict while iterating in PropertiesCodec, this leads to random failures of the py35 unittest gate job. As the logs of a zuul gate job won't be permanent, I wrote a simple script(though may be ugly) to testing this: $ python --version Python 3.5.2 $ cat /tmp/python35_dict_updating.py data_dict = {} def init_dict(): global data_dict for x in range(11): key = 'key' + str(x+1) data_dict[key] = x * 10 def update_dict(): global data_dict for k, v in data_dict.items(): data_dict.update({k: v+1}) def checking_failed(round): global data_dict FAILED = False for x in range(11): key = 'key' + str(x+1) origin = x * 10 current = data_dict[key] if (current - origin) != round: print('%s is %s, expecting %s' % (key, current, origin + round)) FAILED = True break return FAILED TEST_TIMES=5 PASSED = True for x in range(TEST_TIMES): init_dict() for round in range(1, 100): update_dict() if checking_failed(round): print("Failed at round %s" % round) PASSED = False break if PASSED: print("No failures in 5 100-round tests") $ python /tmp/python35_dict_updating.py key9 is 82, expecting 81 Failed at round 1 $ python /tmp/python35_dict_updating.py key2 is 12, expecting 11 Failed at round 1 $ python /tmp/python35_dict_updating.py key9 is 82, expecting 81 Failed at round 1 $ python /tmp/python35_dict_updating.py No failures in 5 100-round tests From the above testing results, we could see it's quite often that one of the item will be updated twice during the dict iteration. This is the reason why test_properties_file_codec failed sometimes. This fix is manually converting the result of dict.items to a list: $ sed -i 's/data_dict.items()/list(data_dict.items())/' \ /tmp/python35_dict_updating.py $ python /tmp/python35_dict_updating.py No failures in 5 100-round tests $ python /tmp/python35_dict_updating.py No failures in 5 100-round tests $ python /tmp/python35_dict_updating.py No failures in 5 100-round tests $ python /tmp/python35_dict_updating.py No failures in 5 100-round tests Closes-Bug: #1764321 Change-Id: Ia9fcfc6519b29f1a9508b79614c5e81456ad57b6 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Fix lower-constraints and uncap eventlet The 'requirements-check' gate job will validate requirements.txt and test-requirements.txt against lower-constraints.txt, i.e. the minimum version should match between lower-constraints and requirements. The change in lower-constraints.txt is produced by fix-lower-constraints.py script from the requirements repo, and 'enum34' is added mannually. This patch also include the commit about 'uncap eventlet' to make the 'requirements-check' actually working, because the change has been already merged in the requirements repo. The original commit message by Doug Hellmann is kept as below for futher referrence: We will manage the eventlet version using constraints now. See the thread starting at http://lists.openstack.org/pipermail/openstack-dev/2018-April/129096.html for more details. Co-Authored-By: Doug Hellmann <doug@doughellmann.com> Change-Id: I61a6904ea433e570420ccd46281162ff174a5737 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Update auth_uri option to www_authenticate_uri Option auth_uri from group keystone_authtoken is deprecated[1]. Use option www_authenticate_uri from group keystone_authtoken. [1]https://review.openstack.org/#/c/508522/ Change-Id: I46bb0acaf1dec9bb4b91d4a2180b3e7ed66741c2 - Merge "Updated from global requirements" - Merge "fix a typo" - Merge "Mox removal for instances_resize API tests" - Merge "add lower-constraints job" - fix a typo Change-Id: Ia7e965a0f94e61e580a4c40eda8dffd8d76d2d64 - Updated from global requirements Change-Id: I9cae682c6b028752340dd0efb18a8980d12ec0b8 - Merge "fix a typo in documentation" - Merge "Revert "Fix false-negative failure report for mysqldump backup"" - fix a typo in documentation Change-Id: Ic53adfb74c628c8ca85b2fa2107999c1ae816833 - add lower-constraints job Create a tox environment for running the unit tests against the lower bounds of the dependencies. Create a lower-constraints.txt to be used to enforce the lower bounds in those tests. Add openstack-tox-lower-constraints job to the zuul configuration. See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html for more details. Change-Id: I5352398603e5755d46c9c033b05af637e379eea0 Depends-On: https://review.openstack.org/555034 Signed-off-by: Doug Hellmann <doug@doughellmann.com> - Mox removal for instances_resize API tests This is the last file using mox3 for testing, so mox3 is also removed from test-requirements.txt. Change-Id: Ie70adc620df04bf4e0ca43ff4e3060b449d2e377 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Revert "Fix false-negative failure report for mysqldump backup" The approach merged in I2c10ae7b6b6094af7819594e889385b0ed77fe4b is not a complete solution, mysqldump will dump warning to stderr too which should not break the backup process. A better fix would be doing checks against the stderr logs just like the InnoBackupEx strategy does. See also: https://bugs.launchpad.net/trove/+bug/1756806 This reverts commit 49fea478ee6d2fe67f6c248b8da9b0ce910184f8. Change-Id: Ia8f938619de48b7c0d2efd6753897335a8ebce41 - Merge "Fix guestagent.test_operating_system for Python3" - Merge "Fix os.path.join() for unittests under python 3.6" - Merge "Add functionality to define different Message and Notification destination" - Merge "Mox removal for MgmtInstance actions API tests" - Merge "Register all replicas in replication scenario test" - Merge "Skip root state inherting scenario tests for Redis" - Merge "Fix client recreation in Redis root-disable test" - Updated from global requirements Change-Id: Ie5b55091c64b7672267cfb052bb3fe2243b95fba - Fix os.path.join() for unittests under python 3.6 Some unittest cases of trove.tests.unittests.guestagent has mocked os.path.expanduser(), the result of the mocked function is passed to os.path.join(). This causes these tests failed under Python3.6 by TypeError, because a str, bytes or os.PathLike object is expected, but a MagicMock object is passed in instead. Giving a string return value for the mocked os.path.expanduser() fixes. Partial-Bug: #1755417 Change-Id: Ieeb7964a65ceb6fab0f20a0c8fefbe38dd25cb10 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Updated from global requirements Change-Id: I79dbf110b618a3bbf82f751dfa4dda0972d087ee - Mox removal for MgmtInstance actions API tests Change-Id: Ibc83ad436db711bd30e4bd771871b5dc349b9e61 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Merge "Avoid diverged slave when migrating MariaDB master" - Updated from global requirements Change-Id: I78782d3acb01c83b402a667905bfe7fbcbd10f4d - Avoid diverged slave when migrating MariaDB master When promoting one slave to the new master in a replication group, previously the old master will be attached to the new one right after the new master is on. For MariaDB, attaching the old master to the new one, new GTID may be created on the old master and also may be synced to some of the other replicas, as they're still connecting to the old master. The new GTID does not exists in the new master, making these slaves diverged from the master. After that, when the diverged slave connects to the new master, 'START SLAVE' will fail with logs like: [ERROR] Error reading packet from server: Error: connecting slave requested to start from GTID X-XXXXXXXXXX-XX, which is not in the master's binlog. Since the master's binlog contains GTIDs with higher sequence numbers, it probably means that the slave has diverged due to executing extra erroneous transactions (server_errno=1236) And these slaves will be left orphan and errored after promote_to_replica_source finishs. Attaching the other replicas to the new master before dealing with the old master will fix this problem and the failure of the trove-scenario-mariadb-multi Zuul job as well. Closes-Bug: #1754539 Change-Id: Ib9c01b07c832f117f712fd613ae55c7de3561116 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Skip root state inherting scenario tests for Redis Redis instances does not inherit root state from backups, and this feature is also not suitable for it, because Redis actually does not have a user account management system. The "root password" for Redis instance is just a configuration parameter for the underlying server. So all scenario tests about checking root state inheritance should be skipped. Further investigations should also be done to the root API for Redis datastore: we may have to delete the RootHistory database record when disabling root, in this way the user could tell the underlying Redis server is not enabling AUTH currently, this could make less confusion for the end users. However if we're going this direction, we should also make it clear that the RootHistory records mean different things for different datastore types. Change-Id: I988fcd906e074e591ccc1c60f09139df94814b5b - Fix client recreation in Redis root-disable test In the Redis root-disable scenario test, when the root user is disabled, a check for ensuring the old password is not valid for connecting to Redis sever fails because connected Redis clients are not affected by the change of the 'requirepass' parameter, they can still issue commands like 'PING'. So when a password is provided, a new client should always be created. Change-Id: I4e948a20f2b67e49c36bac5c20f411d0b84969ef - Merge "Improve code to reduce traverse times" - Register all replicas in replication scenario test In the replication scenario tests, two replicas will be created by a instance create with replica_count equals to 2, but only one of them will be returned in the reponse, so if the testing fails, the debugging logs of the the other one will not be gathered. This patch will fix. The 'replica_2_id' property of ReplicationRunner is also removed in this patch as it's not used at all. Change-Id: I05fca265e6e5f06b515c009203865f0d48acf4dd Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Merge "Fix create mongodb cluster error in multi-network env" - Fix create mongodb cluster error in multi-network env Currently, the code pass "nics=None" when create mongos and config-server during creating mongodb cluster. If we didn't specify the parameter `default_neutron_networks` in trove config file, the value "None" will pass in nova.server.create() eventually, and in a multi network openstack environments, this could raise a “Multiple possible networks found, use a Network ID to be more specific" conflict. This patch fix the bug by specify the value of nics, besides, since the method `validate_instance_nics` has been added[1], we can safely use the first value of instances just like "volume_size". [1] https://review.openstack.org/#/c/522553/ Change-Id: Idf2faf2d619afe132c19d8c9b3c76b1c2c24ec8e Closes-bug: #1715834 - Merge "Fix false-negative failure report for mysqldump backup" - Merge "Fix annotation info error in guestagent-api" - Merge "Return 204 instead of 200 for root-disable API" - Fix annotation info error in guestagent-api Accidental discovery the inaccurate info in guestagent-api annotation about async and sync. So just correct them. Change-Id: Ic544b2e37f766b0c9c9b6d60b9cccf5275d70980 - Fix false-negative failure report for mysqldump backup If mysqldump command has redirect part for stderr, catching error logic by stderr doest not work as expected. Remove redirect part to report BackupError normally for sqldump failure. Change-Id: I2c10ae7b6b6094af7819594e889385b0ed77fe4b Close-Bug: 1753498 - Use neutronclient for floatingip operations As add_floating_ip and remove_floating_ip were removed from novaclient, migrating to neutronclient for floatingip operations. This is the first patch for fixing unittests. As integration tests didn't fail, floatingip operations are not covered there, so it may be necessary to add new scenario tests for this in following patches. Change-Id: Ic41f5898db77a09158e0f8bfa4196bdb5e40b49a Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Return 204 instead of 200 for root-disable API As no content will be returned to the client if a root-disable request succeeds, a HTTP 204 (Not Content) response is more appropriate. Redis root-disable scenario test fails because it's return HTTP 204, but all API related tests are expecting a HTTP 200. Although changing Redis root-disable API is a much simpler way to resolve the problem, migrating from HTTP 200 to HTTP 204 should be a better solution. Related tests and documents are also updated accordingly. APIImpact Change-Id: If732a578009fd35436e810fb7ceceefd1ada3778 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Fix guestagent.test_operating_system for Python3 * Tests for _write_file_as_root failed because tempfile.NamedTemporaryFile use binary mode by default, changing to text mode fixes. * Migrate to olso_serialization.base64 for Base64Codec. Also change the return value from bytearray to bytes for Base64Codec.deserialize. Base64Codec supports reverse encoding(i.e. binary data will be written to the dest file), in this situation, the dest file should be opened with binary mode(and when reading from the file, binary mode should also be used). * stream_codecs.StringConverter converts iterable objects with map function. The behavior of map function is different under Python 2.x and 3.x. However csv.writerows(before Python 3.5) and unpack_singleton in trove.common.utils both need a list object. Converting the return value of map function to a list explicitly before passing to these functions fixes. guestagent.test_operating_system is the last blacklist regex pattern for py3 unittests. With the above problems fixed, blacklist-py3.txt is not needed any more, and tox.ini is also updateted. Migrating from ostestr to stestr is also done while updating tox.ini. Here is the ML post for more information about os-testr and stestr: http://lists.openstack.org/pipermail/openstack-dev/2017-September/122135.html Partially implements: blueprint trove-python3 Change-Id: I31f1f97901d6ebff8a91c1b70a343e724ab806eb Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Merge "Remove entry of policy.json from setup.cfg" - Merge "Use RootHistory to check if root is ever enabled" - Merge "Fix client ping in redis scenario tests" - Merge "[api-ref] Add sections for backups." - Merge "Only launch in-tree zuul jobs when necessary" - Remove entry of policy.json from setup.cfg Review https://review.openstack.org/#/c/506203/ added non existent policy.json(removed in [1]) file in setup.cfg. This patch removes policy.json entry from data_files in setup.cfg. [1] https://review.openstack.org/#/c/509328/ Change-Id: I45afef481aa084f7f9fe716c4768ddf34be848a2 - Use RootHistory to check if root is ever enabled When disabling root, there is no need to call guestagent to check whether the root user is ever enabled. Root hisotry table should be used for this purpose. As datastore specific root controller of MySQL/Cassandra/PostgreSQL were created only for the '_find_root_user' which were calling guestagent to find root user, these controllers are removed and 'DefaultRootController' is used instead. RedisRootController is also updated as it didn't do this check previously. Unittests directory structure is also slightly changed. It's more clear to use similar directory hierarchies for testing and source code, e.g. trove/extensions/common/service.py trove/tests/unitests/extensions/common/test_service.py Change-Id: I9faac61d9650347b51f23e8fcaf5a92aed5fbf93 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Merge "Add default configuration files to data_files" - Merge "Fix incorrect usage of assertTrue" - [api-ref] Add sections for backups. Change-Id: If3a929e9de31dd901c4838e85f91f76941f33d05 Closes-bug: #1663096 Signed-off-by: Fan Zhang <zh.f@outlook.com> - Merge "Fix PostgreSQL non-dynamic configration tests" - Fix client ping in redis scenario tests StrictRedis.ping will return True if the server is online, otherwise raise a ConnectionError exception, so it's not correct to compare the return value with "PONG". Redis manager updates 'requirepass' at runtime only in 'update_overrides' previously, this is incorrect as enable_root/disable_root has been implemented already. This patch adds admin client refreshing in 'apply_overrides' which was removed in [1]. A helper function for Redis root credential is also added RedisHelper. And RedisHelper will cache Redis client, when enable_root/disable_root is requested, cached client should be updated. [1] https://review.openstack.org/#/c/522205/ Change-Id: I558059edbc85f283a02f42920998351cefeb2d08 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Fix PostgreSQL non-dynamic configration tests The parameter 'log_min_duration_statement' was updated to support unit suffixes like 'ms' in [1], but reverted in [2] by accident. [1] https://review.openstack.org/#/c/370982/ [2] https://review.openstack.org/#/c/537665/ Change-Id: I6fa6785bc8438ccc28a815abd1a99526ff03b87c Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Fix incorrect usage of assertTrue Previously assertTrue is incorrectly used as assertEqual or assertIsNotNone, this patch fixes. Some assertEqual around the changes introduced by this patch are also updated to use the proper arguments convention, from assertEqual(observed, expected) to assertEqual(expected, observed) Change-Id: If1d75a850594eff72b69e265a25505382e82022b Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - drop extra word to fix typo Change-Id: I6c55ce58bae79da6d2d18a5fd017d4932a637e17 - Only launch in-tree zuul jobs when necessary Adding 'irrelevant-files' attribute for the in-tree zuul jobs to avoid unnecessary launching. Also add new abstract jobs to hold common attributes among similar jobs. Change-Id: I047412f7fee494e7638f9bdabe0c022619e5fb33 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Remove install-guide tox env The install-guide tox env is not used any more after docs migration. Change-Id: I93c53d02afbbb36d4a3e5641e6d11aab9dcd7d8a - Merge "Allow host URL for versions to be configurable" - Merge "Generate policy sample file automatically." - Merge "report_root should always use context.user" - Generate policy sample file automatically. A new entrypoint in setup.cfg and a config file are added for using olso.policy helper script to generate the sample file. A new tox target also is added to simplify the environment setting up. Now policy sample file can be generated automatically, so the in-repo sample file is no longer needed. Co-Authored-By: Andrew Laski <andrew@lascii.com> Partial-Implements: blueprint policy-in-code Change-Id: Ic336fa154ccc05b5e9db3a8e751a484b1cc5aa9c Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Merge "Add bandit-baseline check job." - Merge "[api-ref]: update instance creating parameters." - Merge "Remove support of creating volume from Nova." - [api-ref]: update instance creating parameters. Closes-Bug: #1643496 Change-Id: I8c9ca4ad690751d3c53cbe9469cf5864a528d631 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Merge "Remove security.authorization option from mongos" - Remove support of creating volume from Nova. Current Nova server volume support is broken. Nova also declared the 'os-volumes_boot' will be deprecated in the future. As creating volumes by cinderclient has been supoorted for a long time, we could just drop support of Nova server volume. This patch also migrate to the new block_device_mapping_v2 parameter of Nova servers creating API. Closes-Bug: #1673408 Change-Id: I74d86241a5a0d0b1804b959313432168f68faf89 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Remove security.authorization option from mongos The "security.authorization" option is only usable for mongod and not for mongos service. When this option is applied to /etc/mongos.conf configuration file, mongos service doesn't start. Change-Id: Id798d7269ee80706002bbebe2ff666fc2fecdd1e Closes-Bug: #1748322 - Update reno for stable/queens Change-Id: I795681aa5045541f8ab852a399ddc1cfa1e7d39e - Add bandit-baseline check job. As bandit-baseline check just call tox to run tests, adding it to in-tree checks instead of gate jobs. Closes-Bug: #1618549 Change-Id: If6848c69e09117789d566ae47977034d5503fa09 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - report_root should always use context.user The RootHistory intends to record by whom and when the root user of the underlying datastore backend was enabled. So the "user" column should always be set to the context user, not the actual root user in the database. But report_root(and report_root_enabled in taskmanager.models) use the database user instead, this is not correct, This patch will fix. Also remove the "user" argument from Root.create() and HistoryRoot.create() because the context is already passed in as an argument when these methods are called. Closes-Bug: #1546372 Change-Id: I3b4c8ee56c7e0876fb384f0c5841d2d391bd555d Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Remove unused optparse code in trove.db add_options in trove.db is not used for a long time, and this is the same situation fofr optparse in python 2.7, so we can safely remove this part from trove.db. Closes-Bug: #1553030 Change-Id: I17228eed5c122bce49ce80ec665fe73d6caa1e04 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Merge "Zuul: Remove project name" - Allow host URL for versions to be configurable The versions resource constructs the links by using application_url, but it's possible that the API endpoint is behind a load balancer or SSL terminator. This means that the application_url might be incorrect. This fix provides a config option (similar to other services) which lets us override the host URL when constructing links for the versions API. Co-Authored-By: Nikhil Manchanda <SlickNik@gmail.com> Change-Id: I23f06c6c2d52ba46c74e0d097c4963d2de731d30 Closes-bug: 1384379 - Zuul: Remove project name Zuul no longer requires the project-name for in-repo configuration. Omitting it makes forking or renaming projects easier. Change-Id: Id890fde91a7e2b3ec9a6cf5ebe540d4fd65938bc - Merge "Update the validation template for postgresql" - Merge "Unable to grow/shrink Vertica 9.x cluster" - Merge "Unable to perform backup on DB2 instance" - Merge "Accept the IBM DB2 license during the DIB process" - Unable to grow/shrink Vertica 9.x cluster During the Vertica cluster growing/shrinking, the "update_vertica" command tries to get AWS metadata even if --ignore-aws-instance-type is specified. The only way to avoid the "_get_macs error" is to define --ignore-install-config option. This is very similar to this bug[1] but this time this for the "trove cluster-grow" and "trove cluster-shrink" commands. [1] https://bugs.launchpad.net/trove/+bug/1745493 Change-Id: I35f2035039f72b3aa2d9beb665096d4fb57e1c4c Closes-Bug: #1747050 - Merge "Improve Vertica 9.x support" - Accept the IBM DB2 license during the DIB process During the DB2 installation, the db2_install process asks a question about the license. Disk Image Builder will hang until "yes" or "no" is manually entered. db2_install script provides the "-y" option to accept automatically the license. From db2_install help: Specifies that you have read and agreed to the license agreement file in the db2/license directory on the CD. This parameter is mandatory when -n is specified. Change-Id: I8a8330adc41cbed60c8472f925b10cf816aed44e Closes-Bug: #1747031 - Merge "[api-ref] Add sections for instance logs." - Merge "Remove hardcoded version in DB2 install path" - Remove hardcoded version in DB2 install path The DB2 path needs to be changed to something basic and not using a version number because we don't want to update the code every time than a new DB2 version is released. Updated DB2 version from 10.5 to 11.1, version 10.5 is no more available on IBM website. Closes-Bug: #1746351 Change-Id: I4fd7c66c3cd5ff8d540e994ada979b59c7def35e - Improve Vertica 9.x support During the Vertica installation, the install script try to get AWS metadata even if --ignore-aws-instance-type is specified. The only way to avoid the _get_macs error is to define --ignore-install-config option. During the cluster creation, if the process is too long, Vertica ask a question: Do you want to continue waiting? (yes/no) [yes] From the adminTools documentation there is no non-interactive or timeout options. The only way to have this question answered is to run "echo yes | " before the "adminTools -t create_db" command. The only available version from Vertica website is 9.0.1. Closes-Bug: #1745493 Change-Id: I94472082fac606f4ff14f26c2ca2b620ef0a3bbb - Merge "Revert Cassandra version to 2 on ubuntu element" - Update the validation template for postgresql Add some configuration options on valuation rules file, these options are availables from the release 9.5 and 9.6 New options from 9.5: - gin_pending_list_limit - idle_in_transaction_session_timeout Changelog: https://www.postgresql.org/docs/10/static/release-9-5.html New options from 9.6: - log_min_duration_statement - backend_flush_after - bgwriter_flush_after - checkpoint_flush_after - force_parallel_mode - parallel_setup_cost - parallel_tuple_cost - replacement_sort_tuples Changelog: https://www.postgresql.org/docs/10/static/release-9-6.html Closes-bug: #1745238 Change-Id: I89d793ccc7fa43deb3c1da7b23ce10cdf1e85c4d - Unable to perform backup on DB2 instance models.DatastoreSchema() needs to be initialized with a database name. This change fix the backup on DB2 instance Change-Id: I95a9c60cd2406d4aa3b6ee52c654b2590a8d7e03 Closes-Bug: #1746626 - Fix gate error Looks like SQLAlchemy in version 1.2.x checks if the boolean value is None, True, False, 1, or 0. We used a "1" in our test, which does not seem to meet the requirements. In the meantime, if we change it to 1 here, our function section needs to be changed as well. Change-Id: I8df2118fee12fd9dd02caf96a23bc59652efbed7 - [api-ref] Add sections for instance logs. Change-Id: I3b0eb2103c89610ec494ac33407e8a7969af1164 Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Merge "Updated from global requirements" - Merge "Add innodb rules for mysql validation template" - Add innodb rules for mysql validation template Add some nodb configuration options on valuation rules file, these options are availables from the release 5.6 and 5.7 https://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html Closes-bug: #1745477 Change-Id: I2636f4fae06e50be9cfee1a344b3d1f094b2173d - Revert Cassandra version to 2 on ubuntu element The element is configured to use version 3 of cassandra, unfortunatly trove is actually not compatible with version 3. Closes-Bug: #1745056 Change-Id: Ic22202eeae6f404433a84def8cf5da0e5dc28c7b - Merge "[api-ref] Update style and instances api." - Merge "Fix Cassandra element" - Fix Cassandra element The pip command was not installed. Added a couple of prerequisites (snappy, Cython) and db management package. The cassandra-tools package provides commands for enabling and disabling incremental backup, snapshots, and many other features. It is provided by the cassandra community. Change-Id: If0f807878d6f3da17e33bc9e40ecb9563de61a25 - Change file permissions on element script Needs to be executable to run Change-Id: I1302d8ea805f3dc023ef7224ed4aa9b4d06ff371 - [api-ref] Update style and instances api. Doc style: * use tildes for heading 2 (following the rst convention); * break source lines exceeded 79 characters (rst convention); * remove unneccessary blank lines: * 4 blank lines between sections; * 2 blank lines between sub-sections; * 1 blank line between paragraphs in a section/subsection; * no blank lines at the bottom of a source file. * add a space after commas in the middle of a line; Instances API: * change the order to match the description at the begin; * add "Update instance name"; * add "Upgrade datastore version". Change-Id: I3520e42f6ad97cb30632cf05241cec316409c9be Signed-off-by: Zhao Chao <zhaochao1984@gmail.com> - Merge "Use neutronclient to get networks." - Merge "[api-ref] Add fault names for the error response codes" - Merge "Remove log translations" - Use neutronclient to get networks. Trove now uses novaclient v2, this patch removes novaclient 1.1 calls. Closes-bug: #1744059 Change-Id: Id7f626cd4ba53dbbb675d38e4dd27db95405f249 - Merge "Missing element-deps files for xenial" - [api-ref] Add fault names for the error response codes Error response codes with fault names will be easier for understanding. Change-Id: I5970cb6d40cf5793f024fd5d46b0da8f24bee0d6 - Remove log translations According to discussions on the ML, log messages should not be translated any more. This patch also: * removes all usage of _LI, _LW, _LE, _LC; * updates log translation hacking rule. ML discussions: http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html The original commit messages are kept as following to make a historic note: remove unwanted log translations recent discussions on the ML have led to the decision to eliminate all translations for messages that are destined for log files but retain them for messages that are going to be returned as exceptions and potentially shown to end users via an API. see [1], [2]. This change does that as follows. 1. If a string is being used to generate an exception, it will still be translated. Still Trove used both _LE and _ for these translations, there are some cases where _LE has been simply changed to _, and not removed. 2. If a string is used for a logging message, remove the use of _, _LE, _LW, _LI. Also, I have long felt that we have had a creep of checks in the pep8 tests that apparently make the code easier to read. I strongly believe that these kinds of "easier to read" things make sense if they are followed across all projects and not just gratuitously added one project at a time. I've taken this opportunity to reduce the merge mess caused by this change, to sync up our ignore flags with a long list of ignores from Nova. When they made the change for removing log translation, they could do it in an automated-way like I have because they didn't have to deal with under and overindented lines for visual edit (E127). Same for Cinder. Changes 448443 [3] and 447839 [4] were inadequate because they only addressed a little part of the problem, namely removing the use of _LE, _LI, and _LW, and I think this is a change we don't need to dribble in a few files at a time. The changes are straightforward and should be taken in a single lump to make it easy to deal with the merges coming. [1] http://lists.openstack.org/pipermail/openstack-operators/2017-March/012887.html [2] http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html [3] https://review.openstack.org/448443 [4] https://review.openstack.org/447839 Co-Authored-By: Amrith Kumar <amrith@amrith.org> Co-Authored-By: Valencia Serrao <vserrao@us.ibm.com> Change-Id: I5f86c982469e625997fc8bd15c6fae0fc77a8c64 - Merge "Fix api exception with unicode tenant name." - Merge "Remove checkpoint_segments validation rules" - Updated from global requirements Change-Id: I875406e1a64dc9f2ed04e6bf3721daf803254f3f - Merge "Adding missing dependencay" - Merge "Fix a error exception code" - Missing element-deps files for xenial The dependency files for cassandra, couchbase and couchdb were missing. Added them. Change-Id: Ib7684dabe41f505c34a598678037cebb4526cae0 - Remove checkpoint_segments validation rules The checkpoint_segments option has been deprecated in PostgreSQL 9.5. Change-Id: Ibc879c0733fb2312d466c592cac2c446fcbf7657 Closes-Bug: #1745034 - Adding missing dependencay For percona on xenial, the element-deps file was missing. Change-Id: I1e44986a5caaab8ddf0a2891a3d692d6f4684153 - Remove use of unsupported TEMPEST_SERVICES variable TEMPEST_SERVICES global variable is not supported by devstack since long back. - I380dd20e5ed716a0bdf92aa02c3730359b8136e4 - I9c24705e494689f09a885eb0a640efd50db33fcf Service availability of tempest known services will be set by devstack with local check. - I02be777bf93143d946ccbb8e9eff637bfd1928d4 This commit removes the unused TEMPEST_SERVICES setting Related-Bug: #1743688 Change-Id: Iccbc833d821a2d7b2bc34860170ce1817fbecead - Fix api exception with unicode tenant name. There are a lot request debug logging in Trove, when some values of headers are encoded in utf8, UnicodeEncodeError will be raised by webob.Request. Override how webob.Request is represented will fix. Closes-Bug: #172012…
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Starting with Python 3.6, invalid escape sequences in string literals are now deprecated. In a future version of Python, invalid escape sequences will be a syntax error. While this deprecation produces a
runtime warning, it only appears if warnings are enabled and the first time the Python source is compiled to byte code. By adding a check to pycodestyle, projects can take advantage of static analysis to catch and fix these future syntax errors.
For more information on the deprecation, see the Python release notes
Fixes #633