Docker部署Dubbo跨主机IP访问解决方案

最近在对现有业务系统进行Dubbo服务化重构，部署方式采用Docker部署，在部署过程中发现Dubbo服务注册的一些问题(因为现在团队中缺少容器化的大神支撑)，在这里和大家进行分享；

Dubbo在Docker中部署时往注册中心注册的是Docker实例的IP地址，通常为：172.18.0.*。这种IP外网是服务访问的。

针对这种方式，在Google上查了很多解决方案，本人觉得比较合理的解决方案：

version: '3'
services:
  server1:
    image: uc-server:1.0
    restart: always
    ports:
      - "8081:8080"
      # 前面为注册到注册中心的端口，后面为docker监听的端口
      - "20881:20881"
    environment:
      # 注册到注册中心的IP，这里我们选择宿主机的IP
      DUBBO_IP_TO_REGISTRY: 116.62.139.15
      # 注册到注册中心的端口
      DUBBO_PORT_TO_REGISTRY: 20881
  server2:
    image: uc-server:1.0
    restart: always
    ports:
      - "8082:8080"
      # 前面为注册到注册中心的端口，后面为docker监听的端口
      - "20882:20882"
    environment:
      # 注册到注册中心的IP，这里我们选择宿主机的IP
      DUBBO_IP_TO_REGISTRY: 116.62.139.15
      # 注册到注册中心的端口
      DUBBO_PORT_TO_REGISTRY: 20882
  server3:
    image: uc-server:1.0
    restart: always
    ports:
      - "8083:8080"
      # 前面为注册到注册中心的端口，后面为docker监听的端口
      - "20883:20883"
    environment:
      # 注册到注册中心的IP，这里我们选择宿主机的IP
      DUBBO_IP_TO_REGISTRY: 116.62.139.15
      # 注册到注册中心的端口
      DUBBO_PORT_TO_REGISTRY: 20883

使用docker-compose进行编排，最主要的两个参数：DUBBO_IP_TO_REGISTRY，DUBBO_PORT_TO_REGISTRY，用于指定注册到注册中心的IP和端口。这样就解决了注册IP的问题。但是在后面测试过程中发现可以通讯，但是一直报错：

com.alibaba.dubbo.remoting.RemotingException: Not found exported service:

通过报错信息查看到Dubbo源码：

boolean isCallBackServiceInvoke = false;
        boolean isStubServiceInvoke = false;
        int port = channel.getLocalAddress().getPort();
        String path = inv.getAttachments().get(Constants.PATH_KEY);
        // if it's callback service on client side
        isStubServiceInvoke = Boolean.TRUE.toString().equals(inv.getAttachments().get(Constants.STUB_EVENT_KEY));
        if (isStubServiceInvoke) {
            port = channel.getRemoteAddress().getPort();
        }
        //callback
        isCallBackServiceInvoke = isClientSide(channel) && !isStubServiceInvoke;
        if (isCallBackServiceInvoke) {
            path = inv.getAttachments().get(Constants.PATH_KEY) + "." + inv.getAttachments().get(Constants.CALLBACK_SERVICE_KEY);
            inv.getAttachments().put(IS_CALLBACK_SERVICE_INVOKE, Boolean.TRUE.toString());
        }
        String serviceKey = serviceKey(port, path, inv.getAttachments().get(Constants.VERSION_KEY), inv.getAttachments().get(Constants.GROUP_KEY));

        DubboExporter<?> exporter = (DubboExporter<?>) exporterMap.get(serviceKey);

        if (exporter == null)
            throw new RemotingException(channel, "Not found exported service: " + serviceKey + " in " + exporterMap.keySet() + ", may be version or group mismatch " + ", channel: consumer: " + channel.getRemoteAddress() + " --> provider: " + channel.getLocalAddress() + ", message:" + inv);

        return exporter.getInvoker();

问题就在获取端口的时候产生的，例如：消费者消费的端口是宿主机对外暴露的20881端口，但是在Docker实例中Dubbo使用的是20880端口，两个端口不一致就造成服务没有暴露的问题。解决这个问题的思路就在与指定Dubbo服务启动时候的暴露端口，但是考虑到扩展性，端口配置不能配置到应用的配置文件中，需要在启动Docker的时候指定暴露的端口，查看Dubbo源码：

com.alibaba.dubbo.config.ServiceConfig.doExportUrlsFor1Protocol

String host = this.findConfigedHosts(protocolConfig, registryURLs, map);
        Integer port = this.findConfigedPorts(protocolConfig, name, map);
        URL url = new URL(name, host, port, (contextPath == null || contextPath.length() == 0 ? "" : contextPath + "/") + path, map);

在源码中可以看到获取暴露端口的方法：

/**
     * Register port and bind port for the provider, can be configured separately
     * Configuration priority: environment variable -> java system properties -> port property in protocol config file
     * -> protocol default port
     *
     * @param protocolConfig
     * @param name
     * @return
     */
    private Integer findConfigedPorts(ProtocolConfig protocolConfig, String name, Map<String, String> map) {
        Integer portToBind = null;

        // parse bind port from environment
        String port = getValueFromConfig(protocolConfig, Constants.DUBBO_PORT_TO_BIND);
        portToBind = parsePort(port);

        // if there's no bind port found from environment, keep looking up.
        if (portToBind == null) {
            portToBind = protocolConfig.getPort();
            if (provider != null && (portToBind == null || portToBind == 0)) {
                portToBind = provider.getPort();
            }
            final int defaultPort = ExtensionLoader.getExtensionLoader(Protocol.class).getExtension(name).getDefaultPort();
            if (portToBind == null || portToBind == 0) {
                portToBind = defaultPort;
            }
            if (portToBind == null || portToBind <= 0) {
                portToBind = getRandomPort(name);
                if (portToBind == null || portToBind < 0) {
                    portToBind = getAvailablePort(defaultPort);
                    putRandomPort(name, portToBind);
                }
                logger.warn("Use random available port(" + portToBind + ") for protocol " + name);
            }
        }

        // save bind port, used as url's key later
        map.put(Constants.BIND_PORT_KEY, String.valueOf(portToBind));

        // registry port, not used as bind port by default
        String portToRegistryStr = getValueFromConfig(protocolConfig, Constants.DUBBO_PORT_TO_REGISTRY);
        Integer portToRegistry = parsePort(portToRegistryStr);
        if (portToRegistry == null) {
            portToRegistry = portToBind;
        }

        return portToRegistry;
    }

整个方法最核心的代码：

String port = getValueFromConfig(protocolConfig, Constants.DUBBO_PORT_TO_BIND);

通过读取系统环境变量：DUBBO_PORT_TO_BIND获取暴露端口。找到了配置，只需要修改下docker-compose.yml就可以指定暴露的端口，所以最终的docker-compose.yml文件是：

version: '3'
services:
  server1:
    image: user-server:1.0
    restart: always
    ports:
      - "8081:8080"
      # 前面为注册到注册中心的端口，后面为docker监听的端口
      - "20881:20881"
    environment:
      # 注册到注册中心的IP，这里我们选择宿主机的IP
      DUBBO_IP_TO_REGISTRY: 116.62.139.15
      # 注册到注册中心的端口
      DUBBO_PORT_TO_REGISTRY: 20881
      DUBBO_PORT_TO_BIND: 20881
  server2:
    image: user-server:1.0
    restart: always
    ports:
      - "8082:8080"
      # 前面为注册到注册中心的端口，后面为docker监听的端口
      - "20882:20882"
    environment:
      # 注册到注册中心的IP，这里我们选择宿主机的IP
      DUBBO_IP_TO_REGISTRY: 116.62.139.15
      # 注册到注册中心的端口
      DUBBO_PORT_TO_REGISTRY: 20882
      DUBBO_PORT_TO_BIND: 20882
  server3:
    image: user-server:1.0
    restart: always
    ports:
      - "8083:8080"
      # 前面为注册到注册中心的端口，后面为docker监听的端口
      - "20883:20883"
    environment:
      # 注册到注册中心的IP，这里我们选择宿主机的IP
      DUBBO_IP_TO_REGISTRY: 116.62.139.15
      # 注册到注册中心的端口
      DUBBO_PORT_TO_REGISTRY: 20883
      DUBBO_PORT_TO_BIND: 20883