Naming module Raft protocol migration (alibaba#3989)

* fix-alibaba#3595, delete the unnecessary code (alibaba#3596)

* [ISSUE alibaba#3566] move the permission code of nacos-core module to nacos-auth module (alibaba#3593)

* move the permission code of nacos-core module to nacos-auth module.

* Fix some code style issues

* address server module auth package name change.

* test change

* Incorrect package name correction

* [ISSUE alibaba#3592] Fix incorrect prompt when accessing the restricted namespace (alibaba#3603)

* Fix incorrect prompt when accessing the restricted namespace

* Modify variable name

* [ISSUE alibaba#3600] Replace the deprecated api of jwt (alibaba#3616)

* replace the deprecated api of jwt

* transfer secretKey to byte array just using String encode with utf-8

* [ISSUE alibaba#3613]  Fix `unit test method not be static` & update publish config listener  in `ConfigTest.java` (alibaba#3614)

* fix `unit test method not be static` & update publish config listener in `ConfigTest.java`

* fix `unit test method not be static` & update publish config listener in `ConfigTest.java`

* move jwt dependency from console,core to auth. (alibaba#3624)

* refactor: unified implementation of http client api adjustment. (alibaba#3639)

* [ISSUE alibaba#3628] set naming client updateTask interval more flexible (alibaba#3637)

* 1.use server cacheMillis event service deleted
2.naming client UpdateTask's interval will inc by failCount that connect with server

* 1.move failCount to updateTask
2.redefine the updateService method name. updateServiceNow -> updateService, wrap updateService in updateServiceNow when first getServiceInfo

* 1.create push client even service is not exist
2.serviceInfo's hosts is empty or can't connect to server both add the updateTsk interval

* format the indent

* fix: create kvstorage

* refactor: create kv storage]

* refactor: refactor rocksdb storage code

* Fix issue 3661 (alibaba#3662)

* improvement: merge upstream/develop

* feat: merge upstream develop

* fix: fix issue alibaba#3661

* [ISSUE#3658] refactor TaskManager and move to nacos-common module (alibaba#3663)

* Add TaskManagerTest

* Move Abstract Task and Task processor to nacos-common

* Add Nacos execute engine interface and nacos task interface

* Refactor Task Manager to NacosDelayTaskExecuteEngine

* for code style

* [ISSUE alibaba#3671] move some tools class into common package (alibaba#3672)

* for alibaba#3621 (alibaba#3668)

* [ISSUE#3315]Nacos client support https (alibaba#3654)

* [ISSUE alibaba#3315] nacos client support https
* common module add tls related classes
* JdkHttpClientRequest support https
* unified IpUtils

* [ISSUE alibaba#3315] nacos client support https
* common module add tls related classes
* JdkHttpClientRequest support https
* unified IpUtils

* [ISSUE alibaba#3315] nacos client support https

* format code

* fix typo and doc format of README file (alibaba#3688)

1. It’s a little weird to use Chinese comma in English
2. Fix typo:   change 'reposity' to  'repository', 'instance' to 'instances'
3. Format other details of the doc

* Try to fix nacos server CLOSE_WAIT (alibaba#3703)

* Fix revert chunk isn't work in Content Comparison page (alibaba#3686)

* fix: fixed cluster node version issue

* Fix Logging in with the wrong username or password will cause jackson serialization results to fail (alibaba#3695) (alibaba#3721)

* [ISSUE#3712] add apache http client factory (alibaba#3716)

* refactor: Add apache http client Factory.

* refactor: Add apache http client Factory.

* add license

* refactor: class name change

* [Issue#3692] Use new distro task engine replace old task dispatcher. (alibaba#3715)

* Add ServiceManagerTest

* Add DistroConsistencyServiceImplTest

* Add new distro entities

* Add new distro sync data change

* add new VerifyTask to do checksum.

* Replace old sync task and checksum task

* Add retry sync change task.

* Fix high cpu load

* For checkstyle

* Combine naming sync task to reduce http cost

* Move some classes

* Refactor keys combined logic

* enhance package construct

* Fix unit test

* For pmd

* [Issue alibaba#3692] Use new distro implmentation to handle sync datum and checksum request (alibaba#3734)

* Add receive data and verify data distro protocol interface

* Use new distro implmentation to handle sync datum and checksum request

* Use new distro implmentation to handle get datum request.

* refactor: adjust the data loading logic when a new protocol is started

* [ISSUE#3692] Use new distro implmentation to handle init all datum request. (alibaba#3744)

* Add load data task in DistroProtocol

* Use new distro implmentation to handle init all datum request.

* Fix unit test

* fix: alibaba#3617 (alibaba#3678)

* fix alibaba#3617

* 调整代码格式

* 修改抛出的Exception类型

* 调整逻辑

* 移除没用到的方法

* 修改注释

* [ISSUE#3658] Move distro sync code to nacos-core module (alibaba#3750)

* Move distro sync code to nacos-core module

* Update unit test

* feat: none

* fix: fixing serialization problems

* replace the deprecated api com.fasterxml.jackson.databind.node.ObjectNode#put(java.lang.String, com.fasterxml.jackson.databind.JsonNode)

* Revert "[alibaba#3368]Cancel empty Long polling thread to improve performance. (#ISSUE3432)" (alibaba#3778)

This reverts commit 95c8bf2.

* [ISSUE alibaba#3658] Some enhance refactor for naming distro (alibaba#3765)

* Some enhance refactor for naming distro

* Remove null code

* [ISSUE alibaba#3687] check serviceName's format(groupName@@serviceName) in server and client (alibaba#3767)

* 1.in server, check serviceName's format 'groupName@@serviceName', groupName and serviceName can't be blank
2.in client, check 'groupName@@serviceName', groupName and serviceName can't be blank

* ignore the check to groupName

* check split's length instead of exception to check argument

* 1.add some notes
2.remove unnecessary code

* modify the notes

* [ISSUE#3790] Supplement http response Content-Encoding processing. (alibaba#3791)

* bug: fix issue alibaba#3790, Supplement http response Content-Encoding processing

* bug: fix issue alibaba#3790, Supplement http response Content-Encoding processing.

* bug: fix issue alibaba#3790, Supplement http response Content-Encoding processing.

* Update ConvertUtils.java (alibaba#3789)

* fix ConvertUtils can not handle FormatException.

* fix: fixed some logic errors

* [ISSUE#3192] naming module replace http client (alibaba#3763)

* naming module replace http client

* refactor: naming module replace http client.

* refactor: naming module replace http client.

* refactor: Add apache http client Factory.

* refactor: naming module replace http client.

* fix code style

* refactor: Add http client config

* refactor: naming module HttpClientManager change

* refactor: naming module HttpClientManager change

* refactor: naming module replace http client.

* fix code style

* refactor: fix JDK http client Use error problem.

* refactor: Query And Header entity init Add non-empty judgment

* Enhance the asynchronous http delete request method to support body passing parameters.

* refactor: apache http client set MaxConnTotal and maxConnPerRoute.

* Fix NullPointerException when no subscriber for slow event (alibaba#3835)

* Set mediaType charset as utf8 (alibaba#3837)

* Fix code style problem in DiskUtils (alibaba#3842)

* Fix http client contentType charset problem (alibaba#3848)

* feature issue alibaba#3804 (alibaba#3805)

* improvement: merge upstream/develop

* feat: merge upstream develop

* feat: manage the loading of configuration files uniformly

* fix: fix copyright style

* style: fix code style

* fix: fix code style

* bug: fix the problem of incorrect judgment of http response code in SubscribeManager#getSubscribers() method (alibaba#3879)

* [ISSUE alibaba#3867] replace the way which get version (alibaba#3872)

* replace the way which get version

* remove version sign application.properties in nacos-api

* [ISSUE alibaba#3871] fix description don't match the error (alibaba#3886)

* Fix jraft problem

* Fix chinese string are truncated in ConcunrrentDiskUtil (alibaba#3883)

* Temp fix raft server can't refresh raft configuration problem

* update spring boot dependencies version (alibaba#3900)

Co-authored-by: yanlinly <yan.lin2009@163.com>
Co-authored-by: 杨翊 SionYang <263976490@qq.com>

* [ISSUE alibaba#3781]Fix service list intermittently lost service (alibaba#3891)

* update service init

* commit futureMap.remove()

* update serviceManager

Co-authored-by: yanlinly <yan.lin2009@163.com>
Co-authored-by: 杨翊 SionYang <263976490@qq.com>

* Fix Listener do not listen new consistency problem

* Fix Performance logger thread call old raft error

* Use datum in new raft processor to compatible old data

* Use multiple kv storage in new raft processor to compatible old data

* Revert PR#2849.

* Move datum key check to KeyBuilder

* Fix 1.3.2 upgrade 1.4.0 can't notify service change problem

* [ISSUE alibaba#3850] ignore socket exception when client destroy already (alibaba#3906)

* if client destroy already, ignore socket exception.

* remove the exception's judgement

* remove unuseful import

* fix: fix merge conflict

* isAddressServerHealth set as true when request success (alibaba#3952)

* [ISSUE#3533] change cache dir with namespace -- part 1: unify the cache dir (alibaba#3859)

* [ISSUE#3533] change cache dir with namespace -- part 2: unify the log dir (alibaba#3882)

* fix typo error (alibaba#3954)

* [ISSUE alibaba#3909] add domain's judgement (alibaba#3913)

* add domain's judgement

* modify domain's judgement, can resolve = true

* remove judgement in 'if' code block

* replace Ip to IP in InetUtils

* log warn info when domain can not be resolved

* fix vaiable name

* 1.fix unit test can't pass (alibaba#3956)

2.update the unit test

* Move remove listener logic to ServiceManager

* Move remove listener logic to ServiceManager

* Fix#3973 (alibaba#3974)

* fix alibaba#3973

* 重复代码抽取到一个方法

* 删除私有方法的注释

* 处理namespace参数的方法提出到一个工具类中

* 修改注释

* 添加licences

* 增加 TenantUtil 的测试

* TenantUtil 改名为 NamespaceUtil

* For alibaba#3384, Fix member extend info do not update error. (alibaba#3982)

* refactor: code refactor

Co-authored-by: 赵延 <1060026287@qq.com>
Co-authored-by: mai.jh <maijh97@gmail.com>
Co-authored-by: ljhrot <296093710@qq.com>
Co-authored-by: Xarrow <zhangjian12424@gmail.com>
Co-authored-by: 杨翊 SionYang <263976490@qq.com>
Co-authored-by: Gagharv <wwfortunate@gmail.com>
Co-authored-by: syapollo <34880947+syapollo@users.noreply.github.com>
Co-authored-by: S2W <joexsue@foxmail.com>
Co-authored-by: Tboy <technotboy@yeah.net>
Co-authored-by: 邪影oO <213539@qq.com>
Co-authored-by: yanlinly <yan.lin2009@163.com>
Co-authored-by: Mark4z <36187602+mark4z@users.noreply.github.com>
Co-authored-by: Marcus <anymre@outlook.com>
Co-authored-by: shizhengxing <798187024@qq.com>
Co-authored-by: ljhrot <ljhrot@qq.com>
Co-authored-by: sanxun0325 <bbz17640380550@163.com>
Co-authored-by: JackSun-Developer <sjtusl@163.com>

api/src/main/java/com/alibaba/nacos/api/naming/utils/NamingUtils.java

@@ -28,6 +28,9 @@
 public class NamingUtils {
 
     public static String getGroupedName(final String serviceName, final String groupName) {
+        if (StringUtils.isBlank(serviceName)) {
+            throw new IllegalArgumentException("Param 'serviceName' is illegal, serviceName is blank");
+        }
         final String resultGroupedName = groupName + Constants.SERVICE_INFO_SPLITER + serviceName;
         return resultGroupedName.intern();
     }

auth/pom.xml

@@ -40,6 +40,10 @@
             <groupId>${project.groupId}</groupId>
             <artifactId>nacos-common</artifactId>
         </dependency>
+        <dependency>
+            <groupId>${project.groupId}</groupId>
+            <artifactId>nacos-sys</artifactId>
+        </dependency>
 
         <dependency>
             <groupId>org.springframework.boot</groupId>

auth/src/main/java/com/alibaba/nacos/auth/common/AuthConfigs.java

@@ -16,12 +16,11 @@
 
 package com.alibaba.nacos.auth.common;
 
-import com.alibaba.nacos.auth.common.env.ReloadableConfigs;
 import com.alibaba.nacos.common.JustForTest;
+import com.alibaba.nacos.sys.utils.ApplicationUtils;
 import io.jsonwebtoken.io.Decoders;
 import org.apache.commons.lang3.BooleanUtils;
 import org.apache.commons.lang3.StringUtils;
-import org.springframework.beans.factory.annotation.Autowired;
 import org.springframework.beans.factory.annotation.Value;
 import org.springframework.context.annotation.Configuration;
 
@@ -40,9 +39,6 @@ public class AuthConfigs {
     @JustForTest
     private static Boolean cachingEnabled = null;
 
-    @Autowired
-    private ReloadableConfigs reloadableConfigs;
-
     /**
      * secret key.
      */
@@ -93,7 +89,7 @@ public boolean isAuthEnabled() {
             return BooleanUtils.toBoolean(enabled);
         }
         return BooleanUtils
-                .toBoolean(reloadableConfigs.getProperties().getProperty("nacos.core.auth.enabled", "false"));
+                .toBoolean(ApplicationUtils.getProperty("nacos.core.auth.enabled", "false"));
     }
 
     /**
@@ -106,7 +102,7 @@ public boolean isCachingEnabled() {
             return cachingEnabled;
         }
         return BooleanUtils
-                .toBoolean(reloadableConfigs.getProperties().getProperty("nacos.core.auth.caching.enabled", "true"));
+                .toBoolean(ApplicationUtils.getProperty("nacos.core.auth.caching.enabled", "true"));
     }
 
     @JustForTest

client/src/main/java/com/alibaba/nacos/client/config/impl/ClientWorker.java

@@ -32,7 +32,6 @@
 import com.alibaba.nacos.client.utils.TenantUtil;
 import com.alibaba.nacos.common.http.HttpRestResult;
 import com.alibaba.nacos.common.lifecycle.Closeable;
-import com.alibaba.nacos.common.utils.ConcurrentHashSet;
 import com.alibaba.nacos.common.utils.ConvertUtils;
 import com.alibaba.nacos.common.utils.MD5Utils;
 import com.alibaba.nacos.common.utils.StringUtils;
@@ -166,7 +165,6 @@ private void removeCache(String dataId, String group) {
             copy.remove(groupKey);
             cacheMap.set(copy);
         }
-        reMakeCacheDataTaskId();
         LOGGER.info("[{}] [unsubscribe] {}", this.agent.getName(), groupKey);
 
         MetricsMonitor.getListenConfigCountMonitor().set(cacheMap.get().size());
@@ -179,38 +177,11 @@ void removeCache(String dataId, String group, String tenant) {
             copy.remove(groupKey);
             cacheMap.set(copy);
         }
-        reMakeCacheDataTaskId();
         LOGGER.info("[{}] [unsubscribe] {}", agent.getName(), groupKey);
 
         MetricsMonitor.getListenConfigCountMonitor().set(cacheMap.get().size());
     }
 
-    /**
-     * Remake cacheData taskId.
-     */
-    private void reMakeCacheDataTaskId() {
-        int listenerSize = cacheMap.get().size();
-        int remakeTaskId = (int) Math.ceil(listenerSize / ParamUtil.getPerTaskConfigSize());
-        if (remakeTaskId < (int) currentLongingTaskCount) {
-            for (int i = 0; i < remakeTaskId; i++) {
-                int count = 0;
-                for (String key : cacheMap.get().keySet()) {
-                    if (count == ParamUtil.getPerTaskConfigSize()) {
-                        break;
-                    }
-                    CacheData cacheData = cacheMap.get().get(key);
-                    cacheData.setTaskId(i);
-                    synchronized (cacheMap) {
-                        Map<String, CacheData> copy = new HashMap<String, CacheData>(this.cacheMap.get());
-                        copy.put(key, cacheData);
-                        cacheMap.set(copy);
-                    }
-                    count++;
-                }
-            }
-        }
-    }
-
     /**
      * Add cache data if absent.
      *
@@ -277,8 +248,6 @@ public CacheData addCacheDataIfAbsent(String dataId, String group, String tenant
                 cache.setInitializing(true);
             } else {
                 cache = new CacheData(configFilterChainManager, agent.getName(), dataId, group, tenant);
-                int taskId = cacheMap.get().size() / (int) ParamUtil.getPerTaskConfigSize();
-                cache.setTaskId(taskId);
                 // fix issue # 1317
                 if (enableRemoteSyncConfig) {
                     String[] ct = getServerConfig(dataId, group, tenant, 3000L);
@@ -425,16 +394,11 @@ public void checkConfigInfo() {
         int longingTaskCount = (int) Math.ceil(listenerSize / ParamUtil.getPerTaskConfigSize());
         if (longingTaskCount > currentLongingTaskCount) {
             for (int i = (int) currentLongingTaskCount; i < longingTaskCount; i++) {
-                taskIdSet.add(i);
                 // The task list is no order.So it maybe has issues when changing.
                 executorService.execute(new LongPollingRunnable(i));
             }
-        } else if (longingTaskCount < currentLongingTaskCount) {
-            for (int i = longingTaskCount; i < (int) currentLongingTaskCount; i++) {
-                taskIdSet.remove(i);
-            }
+            currentLongingTaskCount = longingTaskCount;
         }
-        currentLongingTaskCount = longingTaskCount;
     }
 
     /**
@@ -692,9 +656,7 @@ public void run() {
                 }
                 inInitializingCacheList.clear();
 
-                if (taskIdSet.contains(taskId)) {
-                    executorService.execute(this);
-                }
+                executorService.execute(this);
 
             } catch (Throwable e) {
 
@@ -723,11 +685,6 @@ private void setHealthServer(boolean isHealthServer) {
     private final AtomicReference<Map<String, CacheData>> cacheMap = new AtomicReference<Map<String, CacheData>>(
             new HashMap<String, CacheData>());
 
-    /**
-     * Store the running taskId.
-     */
-    private final ConcurrentHashSet<Integer> taskIdSet = new ConcurrentHashSet<Integer>();
-
     private final HttpAgent agent;
 
     private final ConfigFilterChainManager configFilterChainManager;

client/src/main/java/com/alibaba/nacos/client/logging/AbstractNacosLogging.java

@@ -19,6 +19,8 @@
 import com.alibaba.nacos.common.utils.ConvertUtils;
 import com.alibaba.nacos.common.utils.StringUtils;
 
+import java.io.File;
+
 /**
  * Abstract nacos logging.
  *
@@ -31,13 +33,13 @@ public abstract class AbstractNacosLogging {
 
     private static final String NACOS_LOGGING_DEFAULT_CONFIG_ENABLED_PROPERTY = "nacos.logging.default.config.enabled";
 
-    private static final String NACOS_LOGGING_PATH_PROPERTY = "nacos.logging.path";
+    private static final String NACOS_LOGGING_PATH_PROPERTY = "JM.LOG.PATH";
 
     static {
         String loggingPath = System.getProperty(NACOS_LOGGING_PATH_PROPERTY);
         if (StringUtils.isBlank(loggingPath)) {
             String userHome = System.getProperty("user.home");
-            System.setProperty(NACOS_LOGGING_PATH_PROPERTY, userHome + "/logs/nacos");
+            System.setProperty(NACOS_LOGGING_PATH_PROPERTY, userHome + File.separator + "logs");
         }
     }
 

client/src/main/java/com/alibaba/nacos/client/naming/NacosNamingService.java

@@ -39,6 +39,7 @@
 import com.alibaba.nacos.common.utils.ConvertUtils;
 import com.alibaba.nacos.common.utils.StringUtils;
 
+import java.io.File;
 import java.util.ArrayList;
 import java.util.Iterator;
 import java.util.List;
@@ -151,9 +152,13 @@ private void initLogName(Properties properties) {
     }
 
     private void initCacheDir() {
-        cacheDir = System.getProperty("com.alibaba.nacos.naming.cache.dir");
-        if (StringUtils.isEmpty(cacheDir)) {
-            cacheDir = System.getProperty("user.home") + "/nacos/naming/" + namespace;
+        String jmSnapshotPath = System.getProperty("JM.SNAPSHOT.PATH");
+        if (!StringUtils.isBlank(jmSnapshotPath)) {
+            cacheDir = jmSnapshotPath + File.separator + "nacos" + File.separator + "naming"
+                    + File.separator + namespace;
+        } else {
+            cacheDir = System.getProperty("user.home") + File.separator + "nacos" + File.separator + "naming"
+                    + File.separator + namespace;
         }
     }
 

client/src/main/java/com/alibaba/nacos/client/naming/cache/ConcurrentDiskUtil.java

@@ -143,11 +143,12 @@ public static Boolean writeFileContent(File file, String content, String charset
                 }
             } while (null == lock);
 
-            ByteBuffer sendBuffer = ByteBuffer.wrap(content.getBytes(charsetName));
+            byte[] contentBytes = content.getBytes(charsetName);
+            ByteBuffer sendBuffer = ByteBuffer.wrap(contentBytes);
             while (sendBuffer.hasRemaining()) {
                 channel.write(sendBuffer);
             }
-            channel.truncate(content.length());
+            channel.truncate(contentBytes.length);
         } catch (FileNotFoundException e) {
             throw new IOException("file not exist");
         } finally {

client/src/main/java/com/alibaba/nacos/client/naming/core/PushReceiver.java

@@ -107,6 +107,9 @@ public void run() {
                 udpSocket.send(new DatagramPacket(ack.getBytes(UTF_8), ack.getBytes(UTF_8).length,
                         packet.getSocketAddress()));
             } catch (Exception e) {
+                if (closed) {
+                    return;
+                }
                 NAMING_LOGGER.error("[NA] error while receiving push data", e);
             }
         }

client/src/main/java/com/alibaba/nacos/client/utils/ParamUtil.java

@@ -20,9 +20,9 @@
 import com.alibaba.nacos.api.SystemPropertyKeyConst;
 import com.alibaba.nacos.api.common.Constants;
 import com.alibaba.nacos.common.utils.StringUtils;
+import com.alibaba.nacos.common.utils.VersionUtils;
 import org.slf4j.Logger;
 
-import java.io.InputStream;
 import java.util.Properties;
 import java.util.concurrent.Callable;
 import java.util.regex.Pattern;
@@ -80,19 +80,7 @@ public class ParamUtil {
         }
         LOGGER.info("[settings] [http-client] connect timeout:{}", connectTimeout);
 
-        try {
-            InputStream in = ValidatorUtils.class.getClassLoader().getResourceAsStream("application.properties");
-            Properties props = new Properties();
-            props.load(in);
-            String val = null;
-            val = props.getProperty("version");
-            if (val != null) {
-                clientVersion = val;
-            }
-            LOGGER.info("NACOS_CLIENT_VERSION: {}", clientVersion);
-        } catch (Exception e) {
-            LOGGER.error("[500] read application.properties", e);
-        }
+        clientVersion = VersionUtils.version;
 
         try {
             perTaskConfigSize = Double.valueOf(System.getProperty("PER_TASK_CONFIG_SIZE", "3000"));

client/src/main/resources/nacos-log4j2.xml

@@ -17,8 +17,8 @@
 
 <Configuration status="WARN">
     <Appenders>
-        <RollingFile name="CONFIG_LOG_FILE" fileName="${sys:nacos.logging.path}/config.log"
-            filePattern="${sys:nacos.logging.path}/config.log.%d{yyyy-MM-dd}.%i">
+        <RollingFile name="CONFIG_LOG_FILE" fileName="${sys:JM.LOG.PATH}/nacos/config.log"
+            filePattern="${sys:JM.LOG.PATH}/nacos/config.log.%d{yyyy-MM-dd}.%i">
             <PatternLayout>
                 <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %p [%-5t:%c{2}] %m%n</Pattern>
             </PatternLayout>
@@ -31,8 +31,8 @@
             <DefaultRolloverStrategy max="${sys:JM.LOG.RETAIN.COUNT:-7}"/>
         </RollingFile>
 
-        <RollingFile name="NAMING_LOG_FILE" fileName="${sys:nacos.logging.path}/naming.log"
-            filePattern="${sys:nacos.logging.path}/naming.log.%d{yyyy-MM-dd}.%i">
+        <RollingFile name="NAMING_LOG_FILE" fileName="${sys:JM.LOG.PATH}/nacos/naming.log"
+            filePattern="${sys:JM.LOG.PATH}/nacos/naming.log.%d{yyyy-MM-dd}.%i">
             <PatternLayout>
                 <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %p [%-5t:%c{2}] %m%n</Pattern>
             </PatternLayout>