Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bugfix: sql statement escape symbol #1668

Merged
merged 71 commits into from
Dec 16, 2019
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
817497e
fix insert backticks
jsbxyyx Sep 17, 2019
be130f2
fix insert error
jsbxyyx Sep 17, 2019
b738581
Merge branch 'develop' of https://github.com/seata/seata into fix_ins…
jsbxyyx Sep 18, 2019
78cf0ea
fix: insert column have escape
jsbxyyx Sep 18, 2019
9c40890
refactor: remove unused import
jsbxyyx Sep 18, 2019
735b9e6
refactor: add last char judge
jsbxyyx Sep 18, 2019
04d3cff
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Sep 18, 2019
3c90d65
Merge branch 'develop' into fix_insert_back_quote
xingfudeshi Sep 19, 2019
f1c4422
Merge branch 'develop' into fix_insert_back_quote
xingfudeshi Sep 19, 2019
46112e9
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Sep 21, 2019
bdd8204
refactor: add adapter
jsbxyyx Sep 23, 2019
ecbb869
feature add table meta cache adapter
jsbxyyx Sep 24, 2019
07f620d
refactor: table meta adapter
jsbxyyx Sep 24, 2019
9e58c02
Merge branch 'develop' into fix_insert_back_quote
xingfudeshi Sep 25, 2019
d982074
bugfix: oracle batch insert and insert column has escape
jsbxyyx Sep 25, 2019
8243339
Merge remote-tracking branch 'origin/fix_insert_back_quote' into fix_…
jsbxyyx Sep 25, 2019
59f9bcd
chore: remove unused import
jsbxyyx Sep 25, 2019
83730d3
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Sep 26, 2019
5ff9fdb
fix: batch insert exclude oracle
jsbxyyx Sep 27, 2019
aae2a0f
Merge branch 'develop' into fix_insert_back_quote
xingfudeshi Sep 28, 2019
7d723d3
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Oct 8, 2019
7f3db52
revert
jsbxyyx Oct 8, 2019
dcf4551
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Oct 8, 2019
8aaa5d6
resolved conflicting
jsbxyyx Oct 9, 2019
999b7af
fix conflict
jsbxyyx Oct 10, 2019
a982106
fix conflict
jsbxyyx Oct 10, 2019
dea2375
optimize get tablemeta
jsbxyyx Oct 10, 2019
b1f27d9
optimize get tablemeta
jsbxyyx Oct 10, 2019
801deeb
Merge branch 'develop' into fix_insert_back_quote
slievrly Oct 11, 2019
096d6a2
fix: update origin table meta
jsbxyyx Oct 11, 2019
ec9f944
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Oct 12, 2019
6893cf3
revert
jsbxyyx Oct 12, 2019
89417f3
fix: index name add escape
jsbxyyx Oct 12, 2019
2705542
fix review
jsbxyyx Oct 12, 2019
0c9004e
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Oct 12, 2019
1aab15a
Merge branch 'develop' into fix_insert_back_quote
xingfudeshi Oct 13, 2019
f448ed6
feature: add cache and comment
jsbxyyx Oct 14, 2019
934c097
fix abstract connection proxy
jsbxyyx Oct 14, 2019
11c144b
Merge branch 'develop' into fix_insert_back_quote
slievrly Oct 14, 2019
5132192
fix conflict
jsbxyyx Oct 14, 2019
3170f74
Merge remote-tracking branch 'origin/fix_insert_back_quote' into fix_…
jsbxyyx Oct 14, 2019
a1577df
fix conflict
jsbxyyx Oct 14, 2019
2763760
Merge branch 'develop' into fix_insert_back_quote
slievrly Oct 14, 2019
0cb8e18
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Oct 16, 2019
eeb624e
remove table meta cache decorator
jsbxyyx Oct 16, 2019
07d3366
Merge branch 'develop' of https://github.com/seata/seata into fix_ins…
jsbxyyx Oct 16, 2019
e03c7ee
Merge branch 'develop' into fix_insert_back_quote
xingfudeshi Oct 17, 2019
44feae5
Merge branch 'develop' into fix_insert_back_quote
xingfudeshi Oct 19, 2019
c7ead60
Merge branch 'develop' into fix_insert_back_quote
zjinlei Oct 21, 2019
df9782c
chore: collections judge
jsbxyyx Oct 21, 2019
3aba42b
Merge branch 'develop' into fix_insert_back_quote
zjinlei Oct 22, 2019
0d01b7b
Merge branch 'develop' into fix_insert_back_quote
zjinlei Oct 23, 2019
9d09c86
Merge branch 'develop' into fix_insert_back_quote
zjinlei Oct 24, 2019
0b8e9f7
test: add OracleUndo* case
jsbxyyx Oct 24, 2019
6887792
remove unused imports
jsbxyyx Oct 24, 2019
4b2d894
Merge branch 'develop' into fix_insert_back_quote
zjinlei Oct 24, 2019
bffd8c5
fix test case
jsbxyyx Oct 25, 2019
b325b98
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Nov 1, 2019
9f11cd5
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Nov 11, 2019
6f6f505
fix conflict
jsbxyyx Nov 29, 2019
ff2a9b2
Merge remote-tracking branch 'origin/fix_insert_back_quote' into fix_…
jsbxyyx Nov 29, 2019
22fcecb
fix conflict
jsbxyyx Nov 29, 2019
581cf87
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Dec 6, 2019
6514c1f
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Dec 13, 2019
28b6c84
Merge branch 'develop' into fix_insert_back_quote
zjinlei Dec 15, 2019
b6237e5
restore
jsbxyyx Dec 16, 2019
e13464b
restore
jsbxyyx Dec 16, 2019
8e3aec7
Merge branch 'develop' into fix_insert_back_quote
jsbxyyx Dec 16, 2019
dc7f6b7
fix reviews
jsbxyyx Dec 16, 2019
5afa03c
fix reviews
jsbxyyx Dec 16, 2019
2471817
fix reviews
jsbxyyx Dec 16, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import io.seata.rm.datasource.sql.SQLVisitorFactory;
import io.seata.rm.datasource.sql.struct.TableMeta;
import io.seata.rm.datasource.sql.struct.TableMetaCache;
import io.seata.rm.datasource.sql.struct.TableMetaCacheAdapter;
import io.seata.rm.datasource.sql.struct.TableMetaCacheOracle;

import java.sql.Array;
Expand Down Expand Up @@ -112,14 +113,7 @@ public PreparedStatement prepareStatement(String sql) throws SQLException {
PreparedStatement targetPreparedStatement;
SQLRecognizer sqlRecognizer = SQLVisitorFactory.get(sql, dbType);
if (sqlRecognizer != null && sqlRecognizer.getSQLType() == SQLType.INSERT) {
final boolean oracle = StringUtils.equalsIgnoreCase(JdbcConstants.ORACLE, dbType);
final String tableName = sqlRecognizer.getTableName();
TableMeta tableMeta;
if (oracle) {
tableMeta = TableMetaCacheOracle.getTableMeta(getDataSourceProxy(), tableName);
} else {
tableMeta = TableMetaCache.getTableMeta(getDataSourceProxy(), tableName);
}
TableMeta tableMeta = TableMetaCacheAdapter.getTableMeta(dbType, getDataSourceProxy(), sqlRecognizer.getTableName(), false);
jsbxyyx marked this conversation as resolved.
Show resolved Hide resolved
targetPreparedStatement = getTargetConnection().prepareStatement(sql, new String[]{ tableMeta.getPkName() });
} else {
targetPreparedStatement = getTargetConnection().prepareStatement(sql);
Expand Down
160 changes: 160 additions & 0 deletions rm-datasource/src/main/java/io/seata/rm/datasource/ColumnUtils.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
/*
* Copyright 1999-2019 Seata.io Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.seata.rm.datasource;

import com.alibaba.druid.util.JdbcConstants;
import io.seata.common.util.StringUtils;

import java.util.List;

/**
* column utils
* @author jsbxyyx
* @date 2019/09/17
*/
public final class ColumnUtils {

/**
* The escape
*/
public enum Escape {
/** standard escape */
STANDARD('"'),
/** mysql escape */
MYSQL('`')
;
public final char value;
Escape(char value) {
this.value = value;
}
}

/**
* del escape by db type
* @param cols the cols
* @param dbType the db type
*/
public static void delEscape(List<String> cols, String dbType) {
// sql standard
// https://db.apache.org/derby/docs/10.1/ref/crefsqlj1003454.html
// https://docs.oracle.com/javadb/10.8.3.0/ref/crefsqlj1003454.html
// https://www.informit.com/articles/article.aspx?p=2036581&seqNum=2
delEscape(cols, Escape.STANDARD);
if (StringUtils.equalsIgnoreCase(dbType, JdbcConstants.MYSQL)) {
delEscape(cols, Escape.MYSQL);
}
}

/**
* del escape
* @param cols the cols
* @param escape the escape
* @throws NullPointerException if cols is null
*/
public static void delEscape(List<String> cols, Escape escape) {
if (cols == null) {
throw new NullPointerException("cols is null");
jsbxyyx marked this conversation as resolved.
Show resolved Hide resolved
}
for (int i = 0, len = cols.size(); i < len; i++) {
String col = cols.get(i);
if (col != null && col.charAt(0) == escape.value && col.charAt(col.length() - 1) == escape.value) {
jsbxyyx marked this conversation as resolved.
Show resolved Hide resolved
cols.set(i, delEscape(col, escape));
}
}
}

/**
* del escape by db type
* @param colName the column name
* @param dbType the db type
* @return
*/
public static String delEscape(String colName, String dbType) {
String newColName = delEscape(colName, Escape.STANDARD);
if (StringUtils.equalsIgnoreCase(dbType, JdbcConstants.MYSQL)) {
newColName = delEscape(newColName, Escape.MYSQL);
}
return newColName;
}

/**
* del escape by escape
* @param colName the column name
* @param escape the escape
* @return
*/
public static String delEscape(String colName, Escape escape) {
if (colName == null) {
throw new NullPointerException("colName is null");
}
if (colName.isEmpty()) {
return colName;
}
if (colName.charAt(0) == escape.value && colName.charAt(colName.length() - 1) == escape.value) {
return colName.substring(1, colName.length() - 1);
}
return colName;
}

/**
* add escape by db type
* @param cols the column name list
* @param dbType the db type
*/
public static void addEscape(List<String> cols, String dbType) {
if (cols == null || cols.isEmpty()) {
throw new NullPointerException("cols is null");
jsbxyyx marked this conversation as resolved.
Show resolved Hide resolved
}
for (int i = 0, len = cols.size(); i < len; i++) {
String col = cols.get(i);
if (col != null) {
cols.set(i, addEscape(col, dbType));
}
}
}

/**
* add escape by db type
* @param colName the column name
* @param dbType the db type
* @return the colName left and right add escape
*/
public static String addEscape(String colName, String dbType) {
if (StringUtils.equalsIgnoreCase(dbType, JdbcConstants.MYSQL)) {
return addEscape(colName, ColumnUtils.Escape.MYSQL);
}
return addEscape(colName, ColumnUtils.Escape.STANDARD);
}

/**
* add escape
* @param colName the column name
* @param escape the escape
*/
public static String addEscape(String colName, Escape escape) {
if (colName == null) {
throw new NullPointerException("colName is null");
jsbxyyx marked this conversation as resolved.
Show resolved Hide resolved
}
if (colName.isEmpty()) {
return colName;
}
if (colName.charAt(0) == escape.value && colName.charAt(colName.length() - 1) == escape.value) {
return colName;
}
return String.format("%s%s%s", escape.value, colName, escape.value);
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@
import java.util.List;
import java.util.StringJoiner;

import com.alibaba.druid.util.JdbcConstants;
import io.seata.common.util.CollectionUtils;
import io.seata.common.util.StringUtils;
import io.seata.core.context.RootContext;
Expand All @@ -35,8 +34,7 @@
import io.seata.rm.datasource.sql.WhereRecognizer;
import io.seata.rm.datasource.sql.struct.Field;
import io.seata.rm.datasource.sql.struct.TableMeta;
import io.seata.rm.datasource.sql.struct.TableMetaCache;
import io.seata.rm.datasource.sql.struct.TableMetaCacheOracle;
import io.seata.rm.datasource.sql.struct.TableMetaCacheAdapter;
import io.seata.rm.datasource.sql.struct.TableRecords;
import io.seata.rm.datasource.undo.SQLUndoLog;

Expand Down Expand Up @@ -188,11 +186,7 @@ protected TableMeta getTableMeta(String tableName) {
if (tableMeta != null) {
return tableMeta;
}
if (JdbcConstants.ORACLE.equalsIgnoreCase(statementProxy.getConnectionProxy().getDbType())) {
tableMeta = TableMetaCacheOracle.getTableMeta(statementProxy.getConnectionProxy().getDataSourceProxy(), tableName);
} else {
tableMeta = TableMetaCache.getTableMeta(statementProxy.getConnectionProxy().getDataSourceProxy(), tableName);
}
tableMeta = TableMetaCacheAdapter.getTableMeta(getDbType(), statementProxy.getConnectionProxy().getDataSourceProxy(), tableName);
return tableMeta;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
import com.alibaba.druid.util.JdbcConstants;
import io.seata.common.exception.NotSupportYetException;
import io.seata.common.exception.ShouldNeverHappenException;
import io.seata.rm.datasource.ColumnUtils;
import io.seata.common.util.StringUtils;
import io.seata.rm.datasource.PreparedStatementProxy;
import io.seata.rm.datasource.StatementProxy;
Expand Down Expand Up @@ -104,6 +105,9 @@ protected List<Object> getPkValuesByColumn() throws SQLException {
// insert values including PK
SQLInsertRecognizer recognizer = (SQLInsertRecognizer) sqlRecognizer;
final int pkIndex = getPkIndex();
if (pkIndex == -1) {
throw new ShouldNeverHappenException("pkIndex is " + pkIndex);
}
List<Object> pkValues = null;
if (statementProxy instanceof PreparedStatementProxy) {
PreparedStatementProxy preparedStatementProxy = (PreparedStatementProxy) statementProxy;
Expand All @@ -126,6 +130,10 @@ protected List<Object> getPkValuesByColumn() throws SQLException {
pkValues = new ArrayList<>(rowSize);
for (int i = 0; i < rowSize; i++) {
List<Object> row = insertRows.get(i);
// oracle insert parameter count may be than the actual +1
if (row.isEmpty()) {
continue;
}
Object pkValue = row.get(pkIndex);
int currentRowPlaceholderNum = -1;
for (Object r : row) {
Expand Down Expand Up @@ -219,6 +227,8 @@ protected int getPkIndex() {
String pkName = getTableMeta().getPkName();
List<String> insertColumns = recognizer.getInsertColumns();
if (insertColumns != null && !insertColumns.isEmpty()) {
// add escape
ColumnUtils.addEscape(insertColumns, getDbType());
final int insertColumnsSize = insertColumns.size();
int pkIndex = -1;
for (int paramIdx = 0; paramIdx < insertColumnsSize; paramIdx++) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,14 @@ public void setTableSchemaName(String tableSchemaName) {
this.tableSchemaName = tableSchemaName;
}

/**
* Gets table schema name
* @return
*/
protected String getTableSchemaName() {
return tableSchemaName;
}

/**
* Sets table name.
*
Expand All @@ -117,6 +125,15 @@ public void setTableName(String tableName) {
this.tableName = tableName;
}


/**
* Gets table name
* @return
*/
protected String getTableName() {
return tableName;
}

/**
* Gets column name.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -206,15 +206,6 @@ public void setIndextype(IndexType indextype) {
this.indextype = indextype;
}

/**
* Gets indexvalue.
*
* @return the indexvalue
*/
public List<ColumnMeta> getIndexvalue() {
return values;
}

@Override
public boolean equals(Object o) {
if (this == o) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,13 @@
public class TableMeta {
private String tableName;

/**
* key: column name
*/
private Map<String, ColumnMeta> allColumns = new LinkedHashMap<String, ColumnMeta>();
/**
* key: index name
*/
private Map<String, IndexMeta> allIndexes = new LinkedHashMap<String, IndexMeta>();

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
/*
* Copyright 1999-2019 Seata.io Group.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.seata.rm.datasource.sql.struct;

import io.seata.common.util.CollectionUtils;
import io.seata.rm.datasource.ColumnUtils;

import java.util.List;

/**
* @author jsbxyyx
*/
public class TableMetaAdapter extends TableMeta {

private String dbType;

public String getDbType() {
return dbType;
}

public void setDbType(String dbType) {
this.dbType = dbType;
}

@Override
public void setTableName(String tableName) {
if (tableName == null) {
return ;
}
super.setTableName(ColumnUtils.addEscape(tableName, dbType));
}

@Override
public ColumnMeta getColumnMeta(String colName) {
if (colName == null) {
return null;
}
ColumnMeta col = getAllColumns().get(ColumnUtils.addEscape(colName, dbType));
return col;
}

@Override
public boolean containsPK(List<String> cols) {
if (cols == null) {
return false;
}

List<String> pk = getPrimaryKeyOnlyName();
jsbxyyx marked this conversation as resolved.
Show resolved Hide resolved
if (pk.isEmpty()) {
return false;
}

ColumnUtils.addEscape(cols, dbType);

if (cols.containsAll(pk)) {
jsbxyyx marked this conversation as resolved.
Show resolved Hide resolved
return true;
} else {
return CollectionUtils.toUpperList(cols).containsAll(CollectionUtils.toUpperList(pk));
}
}

}
Loading