Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add W605 warning for invalid escape sequences in string literals #676

Merged
merged 1 commit into from
Sep 27, 2017
Merged

Add W605 warning for invalid escape sequences in string literals #676

merged 1 commit into from
Sep 27, 2017

Conversation

jdufresne
Copy link
Contributor

Starting with Python 3.6, invalid escape sequences in string literals are now deprecated. In a future version of Python, invalid escape sequences will be a syntax error. While this deprecation produces a
runtime warning, it only appears if warnings are enabled and the first time the Python source is compiled to byte code. By adding a check to pycodestyle, projects can take advantage of static analysis to catch and fix these future syntax errors.

For more information on the deprecation, see the Python release notes

A backslash-character pair that is not a valid escape sequence now generates a DeprecationWarning. Although this will eventually become a SyntaxError, that will not be for several Python releases.

Fixes #633

pycodestyle.py Outdated
string = text[start:-len(quote)]

if 'r' not in prefix:
pos = 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not do:

pos = string.find('\\')
while pos >= 0:
    pos += 1
    if string[pos] not in valid:
        yield (...)
    pos = string.find('\\', pos + 1)

Starting with Python 3.6, invalid escape sequences in string literals
are now deprecated. In a future version of Python, invalid escape
sequences will be a syntax error. While this deprecation produces a
runtime warning, it only appears if warnings are enabled and the first
time the Python source is compiled to byte code. By adding a check to
pycodestyle, projects can take advantage of static analysis to catch and
fix these future syntax errors.

For more information on the deprecation, see the Python release notes,
https://docs.python.org/3/whatsnew/3.6.html#deprecated-python-behavior

> A backslash-character pair that is not a valid escape sequence now
> generates a DeprecationWarning. Although this will eventually become a
> SyntaxError, that will not be for several Python releases.

Fixes #633
@jdufresne
Copy link
Contributor Author

Thanks for the review.

I agree, that solution is simpler. I've made the suggested change.

@sigmavirus24 sigmavirus24 merged commit c7448b4 into PyCQA:master Sep 27, 2017
@jdufresne jdufresne deleted the invalid-escape branch November 5, 2017 22:13
jibsheet added a commit to openedx/xqueue that referenced this pull request Apr 12, 2018
powersj pushed a commit to canonical/curtin that referenced this pull request Apr 13, 2018
A bit of information from python doc that I got by never having known:
  String literals may optionally be prefixed with a letter `r' or `R';
  such strings are called raw strings and use different rules for
  backslash escape sequences.
  ...
  Unless an `r' or `R' prefix is present, escape sequences in strings are
  interpreted according to rules similar to those used by Standard C.

So basically, any use of \ not followed by one of [\'"abfnrtv]
or \ooo (octal) \xhh (hex) or a newline is invalid.  This is most
commonly seen for us in regex.  To solve, you either:
 a.) use a raw string r'...'
 b.) correctly escape the \ that was not intended to be interpreted.

Python has deprecated these invalid string literals now
  https://bugs.python.org/issue27364
and pycodestyle is identifying them with a W605 warning.
  PyCQA/pycodestyle#676
smoser added a commit to canonical/cloud-init that referenced this pull request Apr 18, 2018
Python has deprecated these invalid string literals now
  https://bugs.python.org/issue27364
and pycodestyle is identifying them with a W605 warning.
  PyCQA/pycodestyle#676

So basically, any use of \ not followed by one of [\'"abfnrtv]
or \ooo (octal) \xhh (hex) or a newline is invalid.  This is most
comomnly seen for us in regex.  To solve, you either:
 a.) use a raw string r'...'
 b.) correctly escape the \ that was not intended to be interpreted.
Guts added a commit to isogeo/isogeo-2-office that referenced this pull request May 22, 2018
openstack-gerrit pushed a commit to openstack/trove that referenced this pull request Jul 3, 2018
Starting with Python 3.6, invalid escape sequences in string literals
are now deprecated[1]. This influence also automatic style checkers like
pycodestyle which starting with 2.4.0 complains about invalid escape
sequences (W605)[2].

Let's fix all those warnings at once by using raw strings where possible
and adding additional \ where not.

Footnotes:
1 - https://docs.python.org/3/whatsnew/3.6.html#deprecated-python-behavior
2 - PyCQA/pycodestyle#676

Change-Id: I009a366fd8342edfd30890df6fe8e1fca88bf3cc
Signed-off-by: Krzysztof Opasiak <k.opasiak@samsung.com>
openstack-gerrit pushed a commit to openstack/openstack that referenced this pull request Jan 8, 2019
* Update trove from branch 'master'
  - update spelling error
    
    Change-Id: I4d260b00220c999072e193f5289fa1212074dcf9
    
  - Execute functional test jobs running under python3
    
    Functional tests are executed by trovestack script, which uses python
    in default OS version which is 2.7.
    
    This change wraps python in tox and executes trovestack int_tests in
    virtual enviroment, starting them in python3.
    
    Any future python version change could be managed by tox framework.
    
    Change-Id: I3a849978241d3d0669ef9e1c802ff504ac3c32cb
    Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
    
  - Use Ubuntu Xenial distro for integration tests
    
    Default Ubuntu release which is used under devstack, has been changed
    from Xenial to Bionic [1].
    Fortunately, backward compatibility for Xenial was added.
    
    Until we are not compatible with Bionic, we need to use older
    Ubuntu distro.
    
    [1] https://review.openstack.org/#/c/610977/
    
    Change-Id: I66c63cf4fa6e5ac201a0fef1d4a8d26520b3b803
    Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
    
  - Merge "Add missing ws separator between words"
  - Merge "Set Tempest's service_availability setting for Trove"
  - Change openstack-dev to openstack-discuss
    
    Mailinglists have been updated. Openstack-discuss replaces openstack-dev.
    
    Change-Id: I1063113e64f698ff4cb45a4dde78242a16050a37
    
  - Merge "Update http to https"
  - Set Tempest's service_availability setting for Trove
    
    Tempest's service_availability config option includes all the service
    availability which is further used by tests to take decision of skip
    or run the test.
    
    For example, [service_availability].trove is true then, trove test will run
    or if [service_availability].trove is false then, all trove related tests either
    in trove tempest plugin or any other plugins will be skipped.
    
    Currently this setting for trove service[1] is in devstack lib/tempest
    which is being removed by - https://review.openstack.org/#/c/619973/
    
    For better maintenance, we are moving all tempest non-owned service setting
    to service devstack plugin side.
    
    This commit add the setting of trove service on trove devstack plugin.
    
    Closes-Bug: #1743688
    [1] https://git.openstack.org/cgit/openstack/trove-tempest-plugin/tree/trove_tempest_plugin/config.py#n17
    
    Needed-By: https://review.openstack.org/#/c/619973/
    Change-Id: Ie3c6bab6e0be9dfcfe29ec3f412e704f3ad79d61
    
  - Add missing ws separator between words
    
    This is to add missing ws separator between words.
    
    Change-Id: I6debb62be4577a45962f206448a62a14a4199488
    
  - [fix_typos] fix wrongly spell word "configration "
    
    Change-Id: I174ffd6e5799aa62df1d135b595c2958a902560a
    
  - Update http to https
    
    modify http link to https link
    
    Change-Id: I9550e78f3e70885b276f5623fb1cb6b32b6ce892
    
  - Fix the wrong url
    
    Fix the wrong url of HACKING.rst in README.rst
    
    Change-Id: I4dbd1fbfe434612116ddebc02228b00b5ea4a369
    
  - Merge "Add python 3.6 unit test job"
  - Extend contribution doc by IRC contact details
    
    Add details how to contact with trove members over IRC channel.
    
    Change-Id: I00ee60812c6a46f26c7eba24711941d9b5362c96
    
  - Add python 3.6 unit test job
    
    This is a mechanically generated patch to add a unit test job running
    under Python 3.6 as part of the python3-first goal.
    
    See the python3-first goal document for details:
    https://governance.openstack.org/tc/goals/stein/python3-first.html
    
    Change-Id: I2b04e865fb72296f672ab206fcda15e30450db97
    
  - Merge "Fix home direcroty path in trovestack script"
  - Merge "Fix incorrect test_group in zull job definition"
  - Fix incorrect test_group in zull job definition
    
    Both of mysql trove-scenario jobs performed the same tests
    group (mysql-supported-single).
    
    This change sets correct group for trove-scenario-mysql-multi job.
    
    Furthermore, to be consistent, trove_test_datastore has been passed
    explicite for trove-scenario-mysql-single job.
    
    Change-Id: I377e17ebf690c79d1c369116697177f992b3df65
    Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
    
  - Fix home direcroty path in trovestack script
    
    This simple fix solves "no such file or directory" error when user
    home directory is not prefixed by /home.
    
    This is the case, for example in devstack, where stack user home is
    set to /opt/stack/.
    
    Lets use $HOME variable to determine user home location.
    
    Change-Id: I07432d27e1a2035296bc62793fecbe13d0265e7b
    Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
    
  - Fix the conflict of urlparse between python2 and python3
    
    Change-Id: I47d7a524afaa3ae74b7c6c2931d8ebb89438c508
    
  - Merge "Add detailed list for instances"
  - Add trove-status upgrade check command framework
    
    This adds basic framework for trove-status upgrade
    check commands. For now it has only "check_placeholder"
    check implemented.
    Real checks can be added to this tool in the future.
    
    Change-Id: Idfeab4c06cba6f841c17ab6e255a29e8707bfa55
    Story: 2003657
    Task: 26162
    
  - Merge "Increment versioning with pbr instruction"
  - Fix build ubuntu-geust issue on arm64
    
    To build ubuntu-geust on arm64, libffi-dev & libssl-dev are required
    
    Closes-Bug: #1797014
    
    Change-Id: Ia215deac11976fc9fc67e6d823ae11d181bf8083
    Signed-off-by: Zhangfei Gao <zhangfei.gao@linaro.org>
    
  - Increment versioning with pbr instruction
    
    With moving away from required milestone releases, the version numbers
    calculated by PBR on the master branch will not work for those testing
    upgrades from the last stable release. More details can be found in the
    mailing list post here:
    
        http://lists.openstack.org/pipermail/openstack-dev/2018-October/135706.html
    
    This is an empty commit that will cause PBR to increment its calculated
    version to get around this.
    
    PBR will see the following which will cause it to increment the version:
    
    Sem-Ver: feature
    
    Please merge this patch as soon as possible to support those testing
    upgrades.
    
    Change-Id: I2520aaa18a5f7add4356374bb709513669d8d915
    Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
    
  - Be compilance with latest oslo.messaging
    
    Support for the [oslo_messaging_rabbit] section has been removed in
    oslo.messaging==9.0.0 package [1].
    That's why integration tests failures on build devstack step.
    
    This is the moment where we need to use the transport_url directive
    in the [DEFAULT] section instead.
    
    Moreover rpc_backend property (which was used in trove to enable fake
    RPC backend) has been removed from DEFAULT section and API tests
    fails on TIME_OUT as they starts on rabbit RPC backend.
    
    Fake RPC can be now configured as described here [2]:
    transport_url = 'fake:/'
    
    [1] https://docs.openstack.org/releasenotes/oslo.messaging/unreleased.html
    [2] https://docs.openstack.org/oslo.messaging/latest/configuration/conffixture.html
    
    Change-Id: Id6c5a9198d5a213cb085407a1d8b534e7c755f69
    Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
    
  - Add detailed list for instances
    
    Currently, listing instances only allows to get basic information about
    entities. To get the details, one need to query instance "show" endpoint
    for each instance separately. This is inefficient and exposes API to a
    heavier load.
    
    There are use cases in which we want to obtain detailed information
    about all instances. In particular, in services integrating with Trove.
    For example, Vitrage project requires this information to build vertices
    and edges in the resource graph for RCA analysis.
    
    Change-Id: I33252cce41c27cc7302c860dde1f6448ecdf3991
    Signed-off-by: Bartosz Zurkowski <b.zurkowski@samsung.com>
    
  - Merge "Fix cover job"
  - Merge "Fix the mysql start up issue after restore"
  - Merge "Replace 14.04 trusty with 16.04 xenial."
  - Merge "Update character set and coallaction for mysql5.7"
  - Merge "Add blueprints and bugs link in documents"
  - Fix the mysql start up issue after restore
    
    When the trove guest image is built without the proper permission
    setting for /tmp directory in which ini file is created. (could happen
    when using non-official ways of image building), mysqld_safe will fail
    to reset the root password after mysql restore due to AppArmour
    complaints. As a consequence it fails the whole mysql restore procedure.
    
    Changing init-file ownership fixes this problem and makes the backup and
    restore successful.
    
    Co-Authored-By: Jake Yip <jake.yip@unimelb.edu.au>
    
    Closes-Bug: #1791878
    
    Change-Id: Ia164345239197f93fb63697212a9b96c900fbe89
    
  - Fix cover job
    
    The job is currently broken, it runs with testr while
    we switched to stestr. Use the usual setup for coverage as done by
    other projects using stestr in cover.
    
    Previously, we run with concurrency=1 - continue to run in serial mode.
    
    Change-Id: I6e389db5aed8db25b49906b2479af081237fc909
    
  - Merge "Cleanup zuul.yaml"
  - Add blueprints and bugs link in documents
    
    Change-Id: I0d5e1b427506e3965b892b130aca4022619de5ae
    
  - endpoint_type option not used with single tenant
    
    Currently we are not able to specify the endpoint_type
    for Neutron, Nova and Cinder clients with single tenant.
    
    publicURL is configured by default but it could be nice
    to have the possibility to choose anything else.
    
    Change-Id: Ibb791cacc0e08de2d87b4348f84c9e573849ec51
    Closes-Bug: #1776229
    
  - Merge "Add release note link in README"
  - Merge "make tox -e pylint only run pylint"
  - Cleanup zuul.yaml
    
    Small cleanups:
    * Use openstack-cover-jobs template to move cover job to check, remove
      post cover job. Mark it non-voting for now.
    * Use openstack-lower-constraints-jobs, remove jobs
    * Sort list of templates.
    
    Change-Id: I2fad568f253a01a3ca272d2e0af4f747b9682b2c
    
  - add python 3.6 unit test job
    
    This is a mechanically generated patch to add a unit test job running
    under Python 3.6 as part of the python3-first goal.
    
    See the python3-first goal document for details:
    https://governance.openstack.org/tc/goals/stein/python3-first.html
    
    Change-Id: I8b1d6f970057175487253f43fdcbbd9f969fe2a7
    Story: #2002586
    Task: #24342
    
  - switch documentation job to new PTI
    
    This is a mechanically generated patch to switch the documentation
    jobs to use the new PTI versions of the jobs as part of the
    python3-first goal.
    
    See the python3-first goal document for details:
    https://governance.openstack.org/tc/goals/stein/python3-first.html
    
    Also, add docs tox environment so that the job works.
    
    Change-Id: I8dd911d46623c0185d238b293e0218cfa435ed46
    Story: #2002586
    Task: #24342
    
  - Merge "import zuul job settings from project-config"
  - make tox -e pylint only run pylint
    
    Having the pylint environment run redundant commands just makes the
    job take longer.
    
    Change-Id: Ibb86b40e589d1b6f9c3af5901549d7aa5170e8c8
    Signed-off-by: Doug Hellmann <doug@doughellmann.com>
    
  - fix tox python3 overrides
    
    We want to default to running all tox environments under python 3, so
    set the basepython value in each environment.
    
    We do not want to specify a minor version number, because we do not
    want to have to update the file every time we upgrade python.
    
    We do not want to set the override once in testenv, because that
    breaks the more specific versions used in default environments like
    py35 and py36.
    
    Change-Id: I9ee34642c700d1e6ba9c2f3891b7fa1f7f7e1e1d
    Depends-On: I8989fd4798e80eae27408017e1543819a68b4ab1
    Signed-off-by: Doug Hellmann <doug@doughellmann.com>
    Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
    Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
    
  - update pylint to 1.9.2
    
    The older version of pylint being used does not work correctly under
    python 3. In order to be able to update the pylint job to run under
    python 3, we need to update the tool.
    
    This patch updates to the latest version at this time. It also updates
    and pins astroid, which was previously capped. Using a pin instead of
    a cap should let us avoid issues with new releases while being
    specific about which version to actually use.
    
    Disable not-callable because that appears to be a new rule that is
    confused by the use of properties to access things that are set to
    callables.
    
    Co-Authored-By: Fan Zhang <zh.f@outlook.com>
    Co-Authored-By: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
    Change-Id: I65705804b222dcd30a653fe10be3d823fa6143ff
    Signed-off-by: Doug Hellmann <doug@doughellmann.com>
    
  - Use latests version of python-troveclient in tests
    
    This patchset changes version of python-troveclient package version
    used in tests.
    So far, version from global constrain file was used [1].
    
    In order to achive this, a local constrain file for tests requirements
    has been added. This made it possible to override global constrain
    versions.
    
    [1] openstack/requirements/upper-constraints.txt
    
    Change-Id: Ic4383f0249e5bd367461f63f66217af959aefba4
    Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
    
  - import zuul job settings from project-config
    
    This is a mechanically generated patch to complete step 1 of moving
    the zuul job settings out of project-config and into each project
    repository.
    
    Because there will be a separate patch on each branch, the branch
    specifiers for branch-specific jobs have been removed.
    
    Because this patch is generated by a script, there may be some
    cosmetic changes to the layout of the YAML file(s) as the contents are
    normalized.
    
    See the python3-first goal document for details:
    https://governance.openstack.org/tc/goals/stein/python3-first.html
    
    Change-Id: Ida3b1bc70fbd6ab958d8669e925de1d3151f69ff
    Story: #2002586
    Task: #24342
    
  - Replace 14.04 trusty with 16.04 xenial.
    
    Change-Id: If235695625813e24810144f524f0a649f180d3c4
    Closes-Bug: #1780184
    
  - Merge "Enable mutable config in trove"
  - Merge "Remove unused imports from the integration tests"
  - Enable mutable config in trove
    
    New releases of oslo.config support a 'mutable' parameter to Opts.
    oslo.service provides an option here Icec3e664f3fe72614e373b2938e8dee53cf8bc5e
    allows services to tell oslo.service they want mutate_config_files to be
    called by passing a parameter.
    
    This commit is to use the same. This allows trove to benefit from
    I1e7a69de169cc85f4c09954b2f46ce2da7106d90, where the 'debug' option
    (owned by oslo.log) is made mutable. we should be able to turn debug
    logging on and off by changing the config.
    
    tc goal:
    https://governance.openstack.org/tc/goals/rocky/enable-mutable-configuration.html
    
    Co-Authored-By: Anusree A <anusree.a04:@gmail.com>
    Change-Id: I02da8ce44d52b0f0dc67660c8ec76b5effb5f73c
    
  - Merge "Format service apache2 reload section for Debian/Ubuntu"
  - Remove unused imports from the integration tests
    
    Change-Id: I2295b9b9e0435bf459364c2dab7852cc9535edfe
    Co-authored-by: wangqi <wang.qi@99cloud.net>
    Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
    
  - Use print function rather than statement in tests
    
    Python3 does not support the print statement, only the print function.
    Whereas Python 2.6 and 2.7 support both.
    
    Until statement is used, executing pylint via python3 on these files
    causes error.
    
    There is no more old way print in trove repository.
    
    Change-Id: I063eb9dd7e690ac2cdc92810982c7f9a6ae22388
    Signed-off-by: Marcin Piwowarczyk <m.piwowarczy@samsung.com>
    
  - Merge "Reject zero volume size in API"
  - Update reno for stable/rocky
    
    Change-Id: I03f5565b9895deac1a66e2789e163c510a9b936a
    
  - Reject zero volume size in API
    
    According the discussion[1], set volume size to 0 makes no sense.
    This fix reject zero value of volume size with integer or string type.
    The previous commit[2] has reject negative volume size, with this fix,
    value like 0, '0' will be rejected and value like 001, '040' will pass.
    
    [1] https://review.openstack.org/#/c/589034/
    [2] https://review.openstack.org/#/c/164460/
    Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com>
    
    Change-Id: If18a3c69a2815086cc60f9a6a24e6254cc7856df
    
  - Merge "Remove nova conf for guestagent"
  - Merge "Add extended properties support for mongo cluster."
  - Add extended properties support for mongo cluster.
    
    Currently when create a mongodb cluster, mongos and configsvr
    use the volume_size of replica-set node. But mongos and configvr
    are not data node, they don't need volume space as large as data
    node. This patch attend to help user specify the number, the volume
    size and the volume type of mongos/configserver with
    extended_properties[1] argument when creating mongodb. Currently,
    the supported parameters are, num_configsvr, num_mongos,
    configsvr_volume_size, configsvr_volume_type, mongos_volume_size
    and mongos_volume_type.
    
    [1] https://review.openstack.org/#/c/206931/
    
    Closes-Bug: #1734907
    
    Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com>
    Change-Id: Ie48f3961b21f926f983c6713a76b0492952cf4c7
    
  - Remove nova conf for guestagent
    
    It seems that the current guestagent only needs to communicate
    through amqp, and does not need to interact with nova, so the
    related nova configuration can be removed from the guestagent,
    which can reduce some invalid configuration.
    
    Similarly, we removed the related configuration of condutor
    and guestagent in devstack plugin.
    
    Change-Id: I3f0b3b445c053351c1f3e966d77ea228df0e94a7
    
  - Fix replication failure when Swift isn't available
    
      * add Swift token verification in create instance method in Trove API
        when slave_id is given
      * catch ConnectionError exception in verify_swift_auth_token
      * add Swift token verification to guestagent before backup and restore
      * add new exception representing Swift connection error
      * set missing fault information when replication snapshot fails
      * mock verify auth token method in replication and restore unit tests
    
    Closes-Bug: #1395523
    
    Change-Id: I6a21ba2ba890a82875f9b6dae3c6b93bc9fdb4b0
    Signed-off-by: Dariusz Krol <d.krol@samsung.com>
    
  - Merge "More reliable gpg keys importing in DIB elements"
  - Merge "Sync the data fields of DB* class and table fields."
  - Merge "Migrate to Zuul v3 native job definitions"
  - Merge "[doc] Use openstack client command to replace others"
  - Merge "Update auth_uri option to www_authenticate_uri"
  - Merge "Trivial: Update pypi url to new url"
  - Sync the data fields of DB* class and table fields.
    
    This update try to fix some problems:
    1. define the 'id' both in _data_fields and _auto_generated_attrs.
    2. some of tables don't have the 'id' field.
    3. _data_fields of DBInstance miss some table fields.
    4. Change the key of persisted_models as the same of table name to
    make it more readable.
    
    Change-Id: I0ad96824e8de978ff5b6766085549b1e1a2509cf
    Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com>
    
  - Migrate to Zuul v3 native job definitions
    
    This patch migrates the legacy gate jobs to Zuul v3 native ones(a new
    trove-devstack role is created for all those legecy jobs, and the
    previous way of running gate jobs is still kept as we still cannot
    migrate the genade job to a Zuul v3 native one), following the
    documents below:
    https://docs.openstack.org/infra/manual/zuulv3.html
    https://docs.openstack.org/devstack/latest/zuul_ci_jobs_migration.html
    
    Change-Id: I10a1adae3a79b7322118fce180119f0ee5ff2e1b
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - More reliable gpg keys importing in DIB elements
    
    We're still seeing apt-key failed to import gpg keys these days during
    the images building in the gate jobs, the problem is keys.gnupg.net and
    keyserver.ubuntu.com are both not stable according to [1] and [2], it's
    better to adopt pool.sks-keyservers.net instead and with simple retries.
    
    To reduce code duplication, this common apt-key importing function is
    also moved to ubuntu-guest as an environment snippet.
    
    [1] https://www.gnupg.org/faq/gnupg-faq.html#new_user_default_keyserver
    [2] https://sks-keyservers.net/overview-of-pools.php
    
    Closes-Bug: #1579094
    
    Change-Id: I0fe200d140f6f9c4d423dd498797a225e3295a71
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Format service apache2 reload section for Debian/Ubuntu
    
    Change-Id: I5e0c9a8f38a64208eb885086910b60da6a6b4ab7
    
  - Merge "py3.x: Fix usage of gettext.install"
  - [doc] Use openstack client command to replace others
    
    The neutron client commands will be deprecated,
    and 'nova net-list' has already been deprecated, so use
    openstack client commands to replace neutron client.
    
    Change-Id: Idb7d9782bb3af28cba7e318b0ed664641b5d731f
    
  - Merge "Raise timeout for instance resizing checking job"
  - Merge "Add volume_type to apischema"
  - Raise timeout for instance resizing checking job
    
    Recently the trove-functional-mysql usually failed by timeout at
    the test_instance_returns_to_active_after_resize case, and the cause is
    creating and resizing instances could be quit slow(e.g. 1200+ seconds).
    This may be caused by the pressure of the CI system and should be
    further investigated. The patch is a quick fix for the problem by
    raising the timeout to 30 minutes(it's now 900 seconds by default, but
    instead it's 32 minutes for instance creating, so raising the value to
    30 minutes should be reasonable).
    
    Partial-Bug: #1778837
    
    Change-Id: Ib903a72b098499553978ec881843a7bbee25b45f
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Merge "Fix invalid escape sequence warnings"
  - Fix invalid escape sequence warnings
    
    Starting with Python 3.6, invalid escape sequences in string literals
    are now deprecated[1]. This influence also automatic style checkers like
    pycodestyle which starting with 2.4.0 complains about invalid escape
    sequences (W605)[2].
    
    Let's fix all those warnings at once by using raw strings where possible
    and adding additional \ where not.
    
    Footnotes:
    1 - https://docs.python.org/3/whatsnew/3.6.html#deprecated-python-behavior
    2 - https://github.com/PyCQA/pycodestyle/pull/676
    
    Change-Id: I009a366fd8342edfd30890df6fe8e1fca88bf3cc
    Signed-off-by: Krzysztof Opasiak <k.opasiak@samsung.com>
    
  - Merge "change pylint wrapper to let messages be prefixes"
  - Merge "Cleanup testrepository and os-testr requirements"
  - Add release note link in README
    
    Change-Id: I658ade892cd08a1e5a2be8a8a2883f417032b8b9
    
  - Merge "Update Trove's README"
  - py3.x: Fix usage of gettext.install
    
    This needed to be updated to get py3.x to work.
    
    Change-Id: I987854c0266566e1e7722478dbb3e27bcffbb08b
    Closes-Bug: #1777440
    
  - Remove pycrypto from requirements
    
    Remove pycrypto completely from requirements and fix the
    related tests.
    
    Closes-Bug: #1749574
    Depends-On: I5c0c1a238023c116af5a84d899e629f1c7c3513f
    Change-Id: Ibfedd9e2ab0a5e78959108112f57103a089f02d1
    Signed-off-by: Fan Zhang <zh.f@outlook.com>
    
  - change pylint wrapper to let messages be prefixes
    
    The messages for many of the errors reported by pylint have been
    extended to include "suggestions" (especially in the case of undefined
    variables or members). Rather than update all of the individual
    messages in the configuration file, just treat the messages as
    prefixes.
    
    Change-Id: I1f11c50e39dbd43c4a9b54ccaaccf03f5ff07aca
    Signed-off-by: Doug Hellmann <doug@doughellmann.com>
    
  - Update Trove's README
    
    Update the README.rst, change the repo url from github to
    git.openstack.org, add link for trove-sepcs and trove-dashboard,
    add link for CONTRIBUTING.rst and HACKING.rst.
    
    Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com>
    Change-Id: If1c7d3467c7da261ac626325a44aa7eab37ae042
    
  - Add volume_type to apischema
    
    Currently, trove had support pass volume_type during creation of
    instance/cluster, but the schema "volume" only contains the "size",
    it is nesscessary add the "type" to schema "volume". Besides, there
    is a "required: True" under "properities", it is unnecessary.
    
    Change-Id: Id7c4317b0ba62d751956ddb1baee70fe27bc15f4
    Signed-off-by: zhanggang <zhanggang@cmss.chinamobile.com>
    
  - Merge "Switch to cryptography from pycrypto"
  - Add a hook for restore process to check if successful
    
    1.This is the same solution with the backup process. Previously we only
    check the stderr of the restore command, but this method is not
    reliable, and the changes during percona-xtrabackup 2.4.11:
    https://jira.percona.com/browse/PXB-1542
    causes the gate jobs failed in creating a slave MySQL instance.
    
    The new hook currently is only used by InnoBackupEx restore runner with
    checking the exit status and stderr output of xbstream.
    
    2.with[1] merged,this makes DIB_CLOUD_IMAGES more flexible, but it
    break trovestack build image, now we need to specify a more
    detailed path to DIB_CLOUD_IMAGES to get the appropriate content.
    
    [1]:https://review.openstack.org/#/c/568697/
    
    Co-Authored-By: Zhao Chao <zhaochao1984@gmail.com>
    Co-Authored-By: zhanggang <zhanggang@cmss.chinamobile.com>
    Co-Authored-By: jiansong <jian.song@easystack.cn>
    
    Closes-Bug: #1771990
    
    Change-Id: Ibb54d6b5953a775be339fb991a0771961d27eba2
    
  - Switch to cryptography from pycrypto
    
    PyCrypto isn't active developed for quite a while, cryptography is
    recommended instead. This patch does this migration, but still keeps
    pycrytpo as a fallback solution.
    
    Random generation is also migrated to os.urandom as the cryptography
    document suggests:
    https://cryptography.io/en/latest/random-numbers/
    
    Closes-Bug: #1749574
    
    Change-Id: I5c0c1a238023c116af5a84d899e629f1c7c3513f
    Co-Authored-By: Fan Zhang <zh.f@outlook.com>
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Update character set and coallaction for mysql5.7
    
    Our character set and collaction are also kept in the older version
    (possibly mysql5.5), which makes it impossible for us to create new
    character set and collaction through database-create. At the same
    time, we can't display these manually created database.
    
    Update here. More information comes from:
    https://dev.mysql.com/doc/refman/5.7/en/charset-charsets.html
    
    Change-Id: I14c0392e19c7846c7aed1e60fe5b5ca01f22a2f4
    
  - Trivial: Update pypi url to new url
    
    Pypi url changed from [1] to [2]
    
    [1] https://pypi.python.org/pypi/<package>
    [2] https://pypi.org/project/<package>
    
    Change-Id: I3470831e34b3bbeb2fb028b71312f51e5224b502
    
  - Merge "Fix dict iteration in PropertiesCodec"
  - Cleanup testrepository and os-testr requirements
    
    We have migrated to stestr for a while, however the related requirements
    is not updated, this patch does the cleanup, also updates requirements
    for stestr.
    
    Change-Id: I9c8cf2f90ced2261293f91faefc3099740540fe7
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Run unittests under the low-constraints job
    
    The low-constraints job should run unittests by the original desgin, we
    didn't put the commands about running unittests inside the default
    testenv settings, so in fact the low-constraints job did nothing except
    creating a virtualenv.
    
    This patch fixes and also move the unittests commands to the default
    testenv settings like the other OpenStack projects.
    
    Change-Id: Ia344a14a9baf7574b043fc08332869083da6dc1e
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Fix dict iteration in PropertiesCodec
    
    We're updating a dict while iterating in PropertiesCodec, this leads to
    random failures of the py35 unittest gate job.
    
    As the logs of a zuul gate job won't be permanent, I wrote a simple
    script(though may be ugly) to testing this:
    
    $ python --version
    Python 3.5.2
    
    $ cat /tmp/python35_dict_updating.py
    data_dict = {}
    
    def init_dict():
        global data_dict
        for x in range(11):
            key = 'key' + str(x+1)
            data_dict[key] = x * 10
    
    def update_dict():
        global data_dict
        for k, v in data_dict.items():
            data_dict.update({k: v+1})
    
    def checking_failed(round):
        global data_dict
        FAILED = False
        for x in range(11):
            key = 'key' + str(x+1)
            origin = x * 10
            current = data_dict[key]
            if (current - origin) != round:
                print('%s is %s, expecting %s' %
                      (key, current, origin + round))
                FAILED = True
                break
        return FAILED
    
    TEST_TIMES=5
    
    PASSED = True
    for x in range(TEST_TIMES):
        init_dict()
    
        for round in range(1, 100):
            update_dict()
            if checking_failed(round):
                print("Failed at round %s" % round)
                PASSED = False
                break
    
    if PASSED:
        print("No failures in 5 100-round tests")
    
    $ python /tmp/python35_dict_updating.py
    key9 is 82, expecting 81
    Failed at round 1
    $ python /tmp/python35_dict_updating.py
    key2 is 12, expecting 11
    Failed at round 1
    $ python /tmp/python35_dict_updating.py
    key9 is 82, expecting 81
    Failed at round 1
    $ python /tmp/python35_dict_updating.py
    No failures in 5 100-round tests
    
    From the above testing results, we could see it's quite often that one
    of the item will be updated twice during the dict iteration. This is
    the reason why test_properties_file_codec failed sometimes.
    
    This fix is manually converting the result of dict.items to a list:
    
    $ sed -i 's/data_dict.items()/list(data_dict.items())/' \
            /tmp/python35_dict_updating.py
    
    $ python /tmp/python35_dict_updating.py
    No failures in 5 100-round tests
    $ python /tmp/python35_dict_updating.py
    No failures in 5 100-round tests
    $ python /tmp/python35_dict_updating.py
    No failures in 5 100-round tests
    $ python /tmp/python35_dict_updating.py
    No failures in 5 100-round tests
    
    Closes-Bug: #1764321
    Change-Id: Ia9fcfc6519b29f1a9508b79614c5e81456ad57b6
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Fix lower-constraints and uncap eventlet
    
    The 'requirements-check' gate job will validate requirements.txt and
    test-requirements.txt against lower-constraints.txt, i.e. the minimum
    version should match between lower-constraints and requirements. The
    change in lower-constraints.txt is produced by fix-lower-constraints.py
    script from the requirements repo, and 'enum34' is added mannually.
    
    This patch also include the commit about 'uncap eventlet' to make the
    'requirements-check' actually working, because the change has been
    already merged in the requirements repo. The original commit message by
    Doug Hellmann is kept as below for futher referrence:
    
    We will manage the eventlet version using constraints now. See the
    thread starting at
    http://lists.openstack.org/pipermail/openstack-dev/2018-April/129096.html
    for more details.
    
    Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
    
    Change-Id: I61a6904ea433e570420ccd46281162ff174a5737
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Update auth_uri option to www_authenticate_uri
    
    Option auth_uri from group keystone_authtoken is deprecated[1].
    Use option www_authenticate_uri from group keystone_authtoken.
    
    [1]https://review.openstack.org/#/c/508522/
    
    Change-Id: I46bb0acaf1dec9bb4b91d4a2180b3e7ed66741c2
    
  - Merge "Updated from global requirements"
  - Merge "fix a typo"
  - Merge "Mox removal for instances_resize API tests"
  - Merge "add lower-constraints job"
  - fix a typo
    
    Change-Id: Ia7e965a0f94e61e580a4c40eda8dffd8d76d2d64
    
  - Updated from global requirements
    
    Change-Id: I9cae682c6b028752340dd0efb18a8980d12ec0b8
    
  - Merge "fix a typo in documentation"
  - Merge "Revert "Fix false-negative failure report for mysqldump backup""
  - fix a typo in documentation
    
    Change-Id: Ic53adfb74c628c8ca85b2fa2107999c1ae816833
    
  - add lower-constraints job
    
    Create a tox environment for running the unit tests against the lower
    bounds of the dependencies.
    
    Create a lower-constraints.txt to be used to enforce the lower bounds
    in those tests.
    
    Add openstack-tox-lower-constraints job to the zuul configuration.
    
    See http://lists.openstack.org/pipermail/openstack-dev/2018-March/128352.html
    for more details.
    
    Change-Id: I5352398603e5755d46c9c033b05af637e379eea0
    Depends-On: https://review.openstack.org/555034
    Signed-off-by: Doug Hellmann <doug@doughellmann.com>
    
  - Mox removal for instances_resize API tests
    
    This is the last file using mox3 for testing, so mox3 is also removed
    from test-requirements.txt.
    
    Change-Id: Ie70adc620df04bf4e0ca43ff4e3060b449d2e377
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Revert "Fix false-negative failure report for mysqldump backup"
    
    The approach merged in I2c10ae7b6b6094af7819594e889385b0ed77fe4b is not
    a complete solution, mysqldump will dump warning to stderr too which
    should not break the backup process. A better fix would be doing checks
    against the stderr logs just like the InnoBackupEx strategy does.
    
    See also: https://bugs.launchpad.net/trove/+bug/1756806
    
    This reverts commit 49fea478ee6d2fe67f6c248b8da9b0ce910184f8.
    
    Change-Id: Ia8f938619de48b7c0d2efd6753897335a8ebce41
    
  - Merge "Fix guestagent.test_operating_system for Python3"
  - Merge "Fix os.path.join() for unittests under python 3.6"
  - Merge "Add functionality to define different Message and Notification destination"
  - Merge "Mox removal for MgmtInstance actions API tests"
  - Merge "Register all replicas in replication scenario test"
  - Merge "Skip root state inherting scenario tests for Redis"
  - Merge "Fix client recreation in Redis root-disable test"
  - Updated from global requirements
    
    Change-Id: Ie5b55091c64b7672267cfb052bb3fe2243b95fba
    
  - Fix os.path.join() for unittests under python 3.6
    
    Some unittest cases of trove.tests.unittests.guestagent has mocked
    os.path.expanduser(), the result of the mocked function is passed to
    os.path.join(). This causes these tests failed under Python3.6 by
    TypeError, because a str, bytes or os.PathLike object is expected, but a
    MagicMock object is passed in instead. Giving a string return value for
    the mocked os.path.expanduser() fixes.
    
    Partial-Bug: #1755417
    
    Change-Id: Ieeb7964a65ceb6fab0f20a0c8fefbe38dd25cb10
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Updated from global requirements
    
    Change-Id: I79dbf110b618a3bbf82f751dfa4dda0972d087ee
    
  - Mox removal for MgmtInstance actions API tests
    
    Change-Id: Ibc83ad436db711bd30e4bd771871b5dc349b9e61
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Merge "Avoid diverged slave when migrating MariaDB master"
  - Updated from global requirements
    
    Change-Id: I78782d3acb01c83b402a667905bfe7fbcbd10f4d
    
  - Avoid diverged slave when migrating MariaDB master
    
    When promoting one slave to the new master in a replication group,
    previously the old master will be attached to the new one right after
    the new master is on. For MariaDB, attaching the old master to the new
    one, new GTID may be created on the old master and also may be synced
    to some of the other replicas, as they're still connecting to the old
    master. The new GTID does not exists in the new master, making these
    slaves diverged from the master. After that, when the diverged slave
    connects to the new master, 'START SLAVE' will fail with logs like:
    
        [ERROR] Error reading packet from server: Error: connecting slave
        requested to start from GTID X-XXXXXXXXXX-XX, which is not in the
        master's binlog. Since the master's binlog contains GTIDs with
        higher sequence numbers, it probably means that the slave has
        diverged due to executing extra erroneous transactions
        (server_errno=1236)
    
    And these slaves will be left orphan and errored after
    promote_to_replica_source finishs.
    
    Attaching the other replicas to the new master before dealing with the
    old master will fix this problem and the failure of the
    trove-scenario-mariadb-multi Zuul job as well.
    
    Closes-Bug: #1754539
    Change-Id: Ib9c01b07c832f117f712fd613ae55c7de3561116
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Skip root state inherting scenario tests for Redis
    
    Redis instances does not inherit root state from backups, and this
    feature is also not suitable for it, because Redis actually does not
    have a user account management system. The "root password" for Redis
    instance is just a configuration parameter for the underlying server.
    So all scenario tests about checking root state inheritance should be
    skipped.
    
    Further investigations should also be done to the root API for Redis
    datastore: we may have to delete the RootHistory database record when
    disabling root, in this way the user could tell the underlying Redis
    server is not enabling AUTH currently, this could make less confusion
    for the end users. However if we're going this direction, we should
    also make it clear that the RootHistory records mean different things
    for different datastore types.
    
    Change-Id: I988fcd906e074e591ccc1c60f09139df94814b5b
    
  - Fix client recreation in Redis root-disable test
    
    In the Redis root-disable scenario test, when the root user is disabled,
    a check for ensuring the old password is not valid for connecting to
    Redis sever fails because connected Redis clients are not affected by
    the change of the 'requirepass' parameter, they can still issue commands
    like 'PING'. So when a password is provided, a new client should always
    be created.
    
    Change-Id: I4e948a20f2b67e49c36bac5c20f411d0b84969ef
    
  - Merge "Improve code to reduce traverse times"
  - Register all replicas in replication scenario test
    
    In the replication scenario tests, two replicas will be created by a
    instance create with replica_count equals to 2, but only one of them
    will be returned in the reponse, so if the testing fails, the debugging
    logs of the the other one will not be gathered. This patch will fix.
    
    The 'replica_2_id' property of ReplicationRunner is also removed in this
    patch as it's not used at all.
    
    Change-Id: I05fca265e6e5f06b515c009203865f0d48acf4dd
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Merge "Fix create mongodb cluster error in multi-network env"
  - Fix create mongodb cluster error in multi-network env
    
    Currently, the code pass "nics=None" when create mongos and config-server
    during creating mongodb cluster. If we didn't specify the parameter
    `default_neutron_networks` in trove config file, the value "None"
    will pass in nova.server.create() eventually, and in a multi network
    openstack environments, this could raise a “Multiple possible networks
    found, use a Network ID to be more specific" conflict.
    This patch fix the bug by specify the value of nics, besides, since the
    method `validate_instance_nics` has been added[1], we can safely use the
    first value of instances just like "volume_size".
    
    [1] https://review.openstack.org/#/c/522553/
    
    Change-Id: Idf2faf2d619afe132c19d8c9b3c76b1c2c24ec8e
    Closes-bug: #1715834
    
  - Merge "Fix false-negative failure report for mysqldump backup"
  - Merge "Fix annotation info error in guestagent-api"
  - Merge "Return 204 instead of 200 for root-disable API"
  - Fix annotation info error in guestagent-api
    
    Accidental discovery the inaccurate info in guestagent-api
    annotation about async and sync. So just correct them.
    
    Change-Id: Ic544b2e37f766b0c9c9b6d60b9cccf5275d70980
    
  - Fix false-negative failure report for mysqldump backup
    
    If mysqldump command has redirect part for stderr, catching error logic
    by stderr doest not work as expected. Remove redirect part to report
    BackupError normally for sqldump failure.
    
    Change-Id: I2c10ae7b6b6094af7819594e889385b0ed77fe4b
    Close-Bug: 1753498
    
  - Use neutronclient for floatingip operations
    
    As add_floating_ip and remove_floating_ip were removed from novaclient,
    migrating to neutronclient for floatingip operations.
    
    This is the first patch for fixing unittests. As integration tests
    didn't fail, floatingip operations are not covered there, so it may be
    necessary to add new scenario tests for this in following patches.
    
    Change-Id: Ic41f5898db77a09158e0f8bfa4196bdb5e40b49a
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Return 204 instead of 200 for root-disable API
    
    As no content will be returned to the client if a root-disable request
    succeeds, a HTTP 204 (Not Content) response is more appropriate.
    
    Redis root-disable scenario test fails because it's return HTTP 204, but
    all API related tests are expecting a HTTP 200. Although changing Redis
    root-disable API is a much simpler way to resolve the problem, migrating
    from HTTP 200 to HTTP 204 should be a better solution. Related tests and
    documents are also updated accordingly.
    
    APIImpact
    
    Change-Id: If732a578009fd35436e810fb7ceceefd1ada3778
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Fix guestagent.test_operating_system for Python3
    
    * Tests for _write_file_as_root failed because tempfile.NamedTemporaryFile
      use binary mode by default, changing to text mode fixes.
    * Migrate to olso_serialization.base64 for Base64Codec. Also change the
      return value from bytearray to bytes for Base64Codec.deserialize.
      Base64Codec supports reverse encoding(i.e. binary data will be written
      to the dest file), in this situation, the dest file should be opened
      with binary mode(and when reading from the file, binary mode should
      also be used).
    * stream_codecs.StringConverter converts iterable objects with map
      function. The behavior of map function is different under Python 2.x
      and 3.x. However csv.writerows(before Python 3.5) and unpack_singleton in
      trove.common.utils both need a list object. Converting the return
      value of map function to a list explicitly before passing to these
      functions fixes.
    
    guestagent.test_operating_system is the last blacklist regex pattern for
    py3 unittests. With the above problems fixed, blacklist-py3.txt is not
    needed any more, and tox.ini is also updateted.
    
    Migrating from ostestr to stestr is also done while updating tox.ini.
    Here is the ML post for more information about os-testr and stestr:
    
    http://lists.openstack.org/pipermail/openstack-dev/2017-September/122135.html
    
    Partially implements: blueprint trove-python3
    
    Change-Id: I31f1f97901d6ebff8a91c1b70a343e724ab806eb
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Merge "Remove entry of policy.json from setup.cfg"
  - Merge "Use RootHistory to check if root is ever enabled"
  - Merge "Fix client ping in redis scenario tests"
  - Merge "[api-ref] Add sections for backups."
  - Merge "Only launch in-tree zuul jobs when necessary"
  - Remove entry of policy.json from setup.cfg
    
    Review https://review.openstack.org/#/c/506203/ added
    non existent policy.json(removed in [1]) file in setup.cfg.
    
    This patch removes policy.json entry from data_files in
    setup.cfg.
    
    [1] https://review.openstack.org/#/c/509328/
    
    Change-Id: I45afef481aa084f7f9fe716c4768ddf34be848a2
    
  - Use RootHistory to check if root is ever enabled
    
    When disabling root, there is no need to call guestagent to check
    whether the root user is ever enabled. Root hisotry table should
    be used for this purpose.
    
    As datastore specific root controller of MySQL/Cassandra/PostgreSQL
    were created only for the '_find_root_user' which were calling
    guestagent to find root user, these controllers are removed and
    'DefaultRootController' is used instead.
    
    RedisRootController is also updated as it didn't do this check
    previously.
    
    Unittests directory structure is also slightly changed. It's more
    clear to use similar directory hierarchies for testing and source
    code, e.g.
        trove/extensions/common/service.py
        trove/tests/unitests/extensions/common/test_service.py
    
    Change-Id: I9faac61d9650347b51f23e8fcaf5a92aed5fbf93
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Merge "Add default configuration files to data_files"
  - Merge "Fix incorrect usage of assertTrue"
  - [api-ref] Add sections for backups.
    
    Change-Id: If3a929e9de31dd901c4838e85f91f76941f33d05
    Closes-bug: #1663096
    Signed-off-by: Fan Zhang <zh.f@outlook.com>
    
  - Merge "Fix PostgreSQL non-dynamic configration tests"
  - Fix client ping in redis scenario tests
    
    StrictRedis.ping will return True if the server is online, otherwise
    raise a ConnectionError exception, so it's not correct to compare the
    return value with "PONG".
    
    Redis manager updates 'requirepass' at runtime only in
    'update_overrides' previously, this is incorrect as
    enable_root/disable_root has been implemented already. This patch adds
    admin client refreshing in 'apply_overrides' which was removed in [1].
    
    A helper function for Redis root credential is also added RedisHelper.
    And RedisHelper will cache Redis client, when enable_root/disable_root
    is requested, cached client should be updated.
    
    [1] https://review.openstack.org/#/c/522205/
    
    Change-Id: I558059edbc85f283a02f42920998351cefeb2d08
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Fix PostgreSQL non-dynamic configration tests
    
    The parameter 'log_min_duration_statement' was updated to support unit
    suffixes like 'ms' in [1], but reverted in [2] by accident.
    
    [1] https://review.openstack.org/#/c/370982/
    [2] https://review.openstack.org/#/c/537665/
    
    Change-Id: I6fa6785bc8438ccc28a815abd1a99526ff03b87c
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Fix incorrect usage of assertTrue
    
    Previously assertTrue is incorrectly used as assertEqual or
    assertIsNotNone, this patch fixes. Some assertEqual around the changes
    introduced by this patch are also updated to use the proper arguments
    convention, from
        assertEqual(observed, expected)
    to
        assertEqual(expected, observed)
    
    Change-Id: If1d75a850594eff72b69e265a25505382e82022b
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - drop extra word to fix typo
    
    Change-Id: I6c55ce58bae79da6d2d18a5fd017d4932a637e17
    
  - Only launch in-tree zuul jobs when necessary
    
    Adding 'irrelevant-files' attribute for the in-tree zuul jobs to avoid
    unnecessary launching. Also add new abstract jobs to hold common
    attributes among similar jobs.
    
    Change-Id: I047412f7fee494e7638f9bdabe0c022619e5fb33
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Remove install-guide tox env
    
    The install-guide tox env is not used any more after docs migration.
    
    Change-Id: I93c53d02afbbb36d4a3e5641e6d11aab9dcd7d8a
    
  - Merge "Allow host URL for versions to be configurable"
  - Merge "Generate policy sample file automatically."
  - Merge "report_root should always use context.user"
  - Generate policy sample file automatically.
    
    A new entrypoint in setup.cfg and a config file are added for
    using olso.policy helper script to generate the sample file.
    
    A new tox target also is added to simplify the environment
    setting up. Now policy sample file can be generated
    automatically, so the in-repo sample file is no longer needed.
    
    Co-Authored-By: Andrew Laski <andrew@lascii.com>
    Partial-Implements: blueprint policy-in-code
    Change-Id: Ic336fa154ccc05b5e9db3a8e751a484b1cc5aa9c
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Merge "Add bandit-baseline check job."
  - Merge "[api-ref]: update instance creating parameters."
  - Merge "Remove support of creating volume from Nova."
  - [api-ref]: update instance creating parameters.
    
    Closes-Bug: #1643496
    Change-Id: I8c9ca4ad690751d3c53cbe9469cf5864a528d631
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Merge "Remove security.authorization option from mongos"
  - Remove support of creating volume from Nova.
    
    Current Nova server volume support is broken. Nova also declared the
    'os-volumes_boot' will be deprecated in the future. As creating volumes
    by cinderclient has been supoorted for a long time, we could just drop
    support of Nova server volume.
    
    This patch also migrate to the new block_device_mapping_v2 parameter of
    Nova servers creating API.
    
    Closes-Bug: #1673408
    Change-Id: I74d86241a5a0d0b1804b959313432168f68faf89
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Remove security.authorization option from mongos
    
    The "security.authorization" option is only usable for
    mongod and not for mongos service.
    
    When this option is applied to /etc/mongos.conf configuration
    file, mongos service doesn't start.
    
    Change-Id: Id798d7269ee80706002bbebe2ff666fc2fecdd1e
    Closes-Bug: #1748322
    
  - Update reno for stable/queens
    
    Change-Id: I795681aa5045541f8ab852a399ddc1cfa1e7d39e
    
  - Add bandit-baseline check job.
    
    As bandit-baseline check just call tox to run tests, adding it to
    in-tree checks instead of gate jobs.
    
    Closes-Bug: #1618549
    Change-Id: If6848c69e09117789d566ae47977034d5503fa09
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - report_root should always use context.user
    
    The RootHistory intends to record by whom and when the root user of the
    underlying datastore backend was enabled. So the "user" column should
    always be set to the context user, not the actual root user in the
    database. But report_root(and report_root_enabled in
    taskmanager.models) use the database user instead, this is not correct,
    This patch will fix.
    
    Also remove the "user" argument from Root.create() and
    HistoryRoot.create() because the context is already passed in as an
    argument when these methods are called.
    
    Closes-Bug: #1546372
    Change-Id: I3b4c8ee56c7e0876fb384f0c5841d2d391bd555d
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Remove unused optparse code in trove.db
    
    add_options in trove.db is not used for a long time, and this is the
    same situation fofr optparse in python 2.7, so we can safely remove
    this part from trove.db.
    
    Closes-Bug: #1553030
    Change-Id: I17228eed5c122bce49ce80ec665fe73d6caa1e04
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Merge "Zuul: Remove project name"
  - Allow host URL for versions to be configurable
    
    The versions resource constructs the links by using application_url,
    but it's possible that the API endpoint is behind a load balancer
    or SSL terminator. This means that the application_url might be
    incorrect. This fix provides a config option (similar to other
    services) which lets us  override the host URL when constructing
    links for the versions API.
    
    Co-Authored-By: Nikhil Manchanda <SlickNik@gmail.com>
    Change-Id: I23f06c6c2d52ba46c74e0d097c4963d2de731d30
    Closes-bug: 1384379
    
  - Zuul: Remove project name
    
    Zuul no longer requires the project-name for in-repo configuration.
    Omitting it makes forking or renaming projects easier.
    
    Change-Id: Id890fde91a7e2b3ec9a6cf5ebe540d4fd65938bc
    
  - Merge "Update the validation template for postgresql"
  - Merge "Unable to grow/shrink Vertica 9.x cluster"
  - Merge "Unable to perform backup on DB2 instance"
  - Merge "Accept the IBM DB2 license during the DIB process"
  - Unable to grow/shrink Vertica 9.x cluster
    
    During the Vertica cluster growing/shrinking,
    the "update_vertica" command tries to get AWS
    metadata even if --ignore-aws-instance-type is
    specified.
    
    The only way to avoid the "_get_macs error" is to
    define --ignore-install-config option.
    
    This is very similar to this bug[1] but this time
    this for the "trove cluster-grow" and "trove
    cluster-shrink" commands.
    
    [1] https://bugs.launchpad.net/trove/+bug/1745493
    
    Change-Id: I35f2035039f72b3aa2d9beb665096d4fb57e1c4c
    Closes-Bug: #1747050
    
  - Merge "Improve Vertica 9.x support"
  - Accept the IBM DB2 license during the DIB process
    
    During the DB2 installation, the db2_install
    process asks a question about the license. Disk
    Image Builder will hang until "yes" or "no" is
    manually entered.
    
    db2_install script provides the "-y" option to
    accept automatically the license.
    
    From db2_install help:
      Specifies that you have read and agreed to the
      license agreement file in the db2/license
      directory on the CD. This parameter is mandatory
      when -n is specified.
    
    Change-Id: I8a8330adc41cbed60c8472f925b10cf816aed44e
    Closes-Bug: #1747031
    
  - Merge "[api-ref] Add sections for instance logs."
  - Merge "Remove hardcoded version in DB2 install path"
  - Remove hardcoded version in DB2 install path
    
    The DB2 path needs to be changed to something
    basic and not using a version number because we
    don't want to update the code every time than a
    new DB2 version is released.
    
    Updated DB2 version from 10.5 to 11.1, version
    10.5 is no more available on IBM website.
    
    Closes-Bug: #1746351
    Change-Id: I4fd7c66c3cd5ff8d540e994ada979b59c7def35e
    
  - Improve Vertica 9.x support
    
    During the Vertica installation, the install script try to get AWS
    metadata even if --ignore-aws-instance-type is specified.
    
    The only way to avoid the _get_macs error is to define
    --ignore-install-config option.
    
    During the cluster creation, if the process is too long, Vertica
    ask a question:
      Do you want to continue waiting? (yes/no) [yes]
    
    From the adminTools documentation there is no non-interactive or
    timeout options. The only way to have this question answered is
    to run "echo yes | " before the "adminTools -t create_db" command.
    
    The only available version from Vertica website is 9.0.1.
    
    Closes-Bug: #1745493
    Change-Id: I94472082fac606f4ff14f26c2ca2b620ef0a3bbb
    
  - Merge "Revert Cassandra version to 2 on ubuntu element"
  - Update the validation template for postgresql
    
    Add some configuration options on valuation rules file,
    these options are availables from the release 9.5 and 9.6
    
    New options from 9.5:
     - gin_pending_list_limit
     - idle_in_transaction_session_timeout
    Changelog:
    https://www.postgresql.org/docs/10/static/release-9-5.html
    
    New options from 9.6:
     - log_min_duration_statement
     - backend_flush_after
     - bgwriter_flush_after
     - checkpoint_flush_after
     - force_parallel_mode
     - parallel_setup_cost
     - parallel_tuple_cost
     - replacement_sort_tuples
    Changelog:
    https://www.postgresql.org/docs/10/static/release-9-6.html
    
    Closes-bug: #1745238
    Change-Id: I89d793ccc7fa43deb3c1da7b23ce10cdf1e85c4d
    
  - Unable to perform backup on DB2 instance
    
    models.DatastoreSchema() needs to be initialized
    with a database name.
    
    This change fix the backup on DB2 instance
    
    Change-Id: I95a9c60cd2406d4aa3b6ee52c654b2590a8d7e03
    Closes-Bug: #1746626
    
  - Fix gate error
    
    Looks like SQLAlchemy in version 1.2.x checks if the boolean value
    is None, True, False, 1, or 0. We used a "1" in our test, which
    does not seem to meet the requirements. In the meantime, if we
    change it to 1 here, our function section needs to be changed
    as well.
    
    Change-Id: I8df2118fee12fd9dd02caf96a23bc59652efbed7
    
  - [api-ref] Add sections for instance logs.
    
    Change-Id: I3b0eb2103c89610ec494ac33407e8a7969af1164
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Merge "Updated from global requirements"
  - Merge "Add innodb rules for mysql validation template"
  - Add innodb rules for mysql validation template
    
    Add some nodb configuration options on valuation rules file,
    these options are availables from the release 5.6 and 5.7
    
    https://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html
    https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html
    
    Closes-bug: #1745477
    Change-Id: I2636f4fae06e50be9cfee1a344b3d1f094b2173d
    
  - Revert Cassandra version to 2 on ubuntu element
    
    The element is configured to use version 3 of cassandra, unfortunatly
    trove is actually not compatible with version 3.
    
    Closes-Bug: #1745056
    Change-Id: Ic22202eeae6f404433a84def8cf5da0e5dc28c7b
    
  - Merge "[api-ref] Update style and instances api."
  - Merge "Fix Cassandra element"
  - Fix Cassandra element
    
    The pip command was not installed.  Added a couple of
    prerequisites (snappy, Cython) and db management package.
    The cassandra-tools package provides commands for enabling
    and disabling incremental backup, snapshots, and many other
    features.  It is provided by the cassandra community.
    
    Change-Id: If0f807878d6f3da17e33bc9e40ecb9563de61a25
    
  - Change file permissions on element script
    
    Needs to be executable to run
    
    Change-Id: I1302d8ea805f3dc023ef7224ed4aa9b4d06ff371
    
  - [api-ref] Update style and instances api.
    
    Doc style:
    * use tildes for heading 2 (following the rst convention);
    * break source lines exceeded 79 characters (rst convention);
    * remove unneccessary blank lines:
      * 4 blank lines between sections;
      * 2 blank lines between sub-sections;
      * 1 blank line between paragraphs in a section/subsection;
      * no blank lines at the bottom of a source file.
    * add a space after commas in the middle of a line;
    
    Instances API:
    * change the order to match the description at the begin;
    * add "Update instance name";
    * add "Upgrade datastore version".
    
    Change-Id: I3520e42f6ad97cb30632cf05241cec316409c9be
    Signed-off-by: Zhao Chao <zhaochao1984@gmail.com>
    
  - Merge "Use neutronclient to get networks."
  - Merge "[api-ref] Add fault names for the error response codes"
  - Merge "Remove log translations"
  - Use neutronclient to get networks.
    
    Trove now uses novaclient v2, this patch removes novaclient 1.1 calls.
    
    Closes-bug: #1744059
    Change-Id: Id7f626cd4ba53dbbb675d38e4dd27db95405f249
    
  - Merge "Missing element-deps files for xenial"
  - [api-ref] Add fault names for the error response codes
    
    Error response codes with fault names will be easier for understanding.
    
    Change-Id: I5970cb6d40cf5793f024fd5d46b0da8f24bee0d6
    
  - Remove log translations
    
    According to discussions on the ML, log messages should not be
    translated any more. This patch also:
    * removes all usage of _LI, _LW, _LE, _LC;
    * updates log translation hacking rule.
    
    ML discussions:
    http://lists.openstack.org/pipermail/openstack-i18n/2016-November/002574.html
    http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html
    
    The original commit messages are kept as following to make a historic
    note:
    
    remove unwanted log translations
    
    recent discussions on the ML have led to the decision to eliminate all
    translations for messages that are destined for log files but retain
    them for messages that are going to be returned as exceptions and
    potentially shown to end users via an API. see [1], [2].
    
    This change does that as follows.
    
    1. If a string is being used to generate an exception, it will still
    be translated. Still Trove used both _LE and _ for these translations,
    there are some cases where _LE has been simply changed to _, and not
    removed.
    
    2. If a string is used for a logging message, remove the use of _,
    _LE, _LW, _LI.
    
    Also, I have long felt that we have had a creep of checks in the pep8
    tests that apparently make the code easier to read. I strongly believe
    that these kinds of "easier to read" things make sense if they are
    followed across all projects and not just gratuitously added one
    project at a time.
    
    I've taken this opportunity to reduce the merge mess caused by this
    change, to sync up our ignore flags with a long list of ignores from
    Nova. When they made the change for removing log translation, they
    could do it in an automated-way like I have because they didn't have
    to deal with under and overindented lines for visual edit (E127). Same
    for Cinder.
    
    Changes 448443 [3] and 447839 [4] were inadequate because they only
    addressed a little part of the problem, namely removing the use of
    _LE, _LI, and _LW, and I think this is a change we don't need to
    dribble in a few files at a time. The changes are straightforward and
    should be taken in a single lump to make it easy to deal with the
    merges coming.
    
    [1] http://lists.openstack.org/pipermail/openstack-operators/2017-March/012887.html
    [2] http://lists.openstack.org/pipermail/openstack-dev/2017-March/113365.html
    [3] https://review.openstack.org/448443
    [4] https://review.openstack.org/447839
    
    Co-Authored-By: Amrith Kumar <amrith@amrith.org>
    Co-Authored-By: Valencia Serrao <vserrao@us.ibm.com>
    Change-Id: I5f86c982469e625997fc8bd15c6fae0fc77a8c64
    
  - Merge "Fix api exception with unicode tenant name."
  - Merge "Remove checkpoint_segments validation rules"
  - Updated from global requirements
    
    Change-Id: I875406e1a64dc9f2ed04e6bf3721daf803254f3f
    
  - Merge "Adding missing dependencay"
  - Merge "Fix a error exception code"
  - Missing element-deps files for xenial
    
    The dependency files for cassandra, couchbase and couchdb
    were missing. Added them.
    
    Change-Id: Ib7684dabe41f505c34a598678037cebb4526cae0
    
  - Remove checkpoint_segments validation rules
    
    The checkpoint_segments option has been deprecated in
    PostgreSQL 9.5.
    
    Change-Id: Ibc879c0733fb2312d466c592cac2c446fcbf7657
    Closes-Bug: #1745034
    
  - Adding missing dependencay
    
    For percona on xenial, the element-deps file was missing.
    
    Change-Id: I1e44986a5caaab8ddf0a2891a3d692d6f4684153
    
  - Remove use of unsupported TEMPEST_SERVICES variable
    
    TEMPEST_SERVICES global variable is not supported
    by devstack since long back.
    - I380dd20e5ed716a0bdf92aa02c3730359b8136e4
    - I9c24705e494689f09a885eb0a640efd50db33fcf
    
    Service availability of tempest known services will be
    set by devstack with local check.
    - I02be777bf93143d946ccbb8e9eff637bfd1928d4
    
    This commit removes the unused TEMPEST_SERVICES setting
    Related-Bug: #1743688
    
    Change-Id: Iccbc833d821a2d7b2bc34860170ce1817fbecead
    
  - Fix api exception with unicode tenant name.
    
    There are a lot request debug logging in Trove, when some values of
    headers are encoded in utf8, UnicodeEncodeError will be raised by
    webob.Request. Override how webob.Request is represented will fix.
    
    Closes-Bug: #172012…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

detect invalid escape sequences
2 participants