running initial pings clientQA ping -------------------- nmap found 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.6 IS UP fd00:6b13:d07c:4acf::6 IS UP 10.116.0.2 IS UP 10.116.0.3 IS UP 10.116.0.4 IS UP 10.116.0.5 IS UP 10.116.0.6 IS DOWN 10.116.0.7 IS DOWN 10.116.0.8 IS UP 10.116.0.9 IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP docker-netclient ping ----------------- nmap found arm-redhat ping ----------------- nmap found 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.6 IS UP fd00:6b13:d07c:4acf::6 IS UP 10.116.0.2 IS UP 10.116.0.3 IS UP 10.116.0.4 IS UP 10.116.0.5 IS UP 10.116.0.6 IS DOWN 10.116.0.7 IS DOWN 10.116.0.8 IS UP 10.116.0.9 IS UP 10.233.54.9 IS DOWN fd00:6b13:d07c:4acf::9 IS DOWN 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP userspace-wireguard ping ----------------- nmap found 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP clientqa02 ping ----------------- nmap found 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP clientqa03 ping ----------------- nmap found clientqa04 ping ----------------- nmap found 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.9 IS DOWN fd00:6b13:d07c:4acf::9 IS DOWN 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP checking routing clientQA ip route -------------------- default via 165.22.112.1 dev eth0 proto static 10.16.0.0/16 dev eth0 proto kernel scope link src 10.16.0.10 10.106.0.0/20 dev eth1 proto kernel scope link src 10.106.0.9 10.116.0.0/16 dev netmaker src 10.233.54.8 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.8 165.22.112.0/20 dev eth0 proto kernel scope link src 165.22.115.179 ::1 dev lo proto kernel metric 256 pref medium 2a03:b0c0:1:d0::/64 dev eth0 proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via 2a03:b0c0:1:d0::1 dev eth0 proto static metric 1024 pref medium docker-netclient ip route -------------------- default via 170.64.224.1 dev eth0 proto static 10.49.0.0/16 dev eth0 proto kernel scope link src 10.49.0.7 10.126.0.0/20 dev eth1 proto kernel scope link src 10.126.0.2 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.3 170.64.224.0/20 dev eth0 proto kernel scope link src 170.64.238.167 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown ::1 dev lo proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev docker0 proto kernel metric 256 linkdown pref medium arm-redhat ip route -------------------- default via 172.31.16.1 dev eth0 proto dhcp src 172.31.23.197 metric 100 10.116.0.0/16 dev netmaker src 10.233.54.2 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.2 172.31.16.0/20 dev eth0 proto kernel scope link src 172.31.23.197 metric 100 ::1 dev lo proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium userspace-wireguard ip route -------------------- default via 24.144.80.1 dev eth0 proto static 10.48.0.0/16 dev eth0 proto kernel scope link src 10.48.0.14 10.116.0.0/16 dev netmaker src 10.233.54.4 10.124.0.0/20 dev eth1 proto kernel scope link src 10.124.0.13 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.4 24.144.80.0/20 dev eth0 proto kernel scope link src 24.144.84.150 ::1 dev lo proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium clientqa02 ip route -------------------- default via 165.22.112.1 dev eth0 proto static 10.16.0.0/16 dev eth0 proto kernel scope link src 10.16.0.11 10.106.0.0/20 dev eth1 proto kernel scope link src 10.106.0.10 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.5 98.216.56.86 via 165.22.112.1 dev eth0 159.223.160.241 via 165.22.112.1 dev eth0 165.22.112.0/20 dev eth0 proto kernel scope link src 165.22.115.252 165.22.115.179 via 165.22.112.1 dev eth0 209.38.230.81 via 165.22.112.1 dev eth0 ::1 dev lo proto kernel metric 256 pref medium 2a03:b0c0:1:d0::/64 dev eth0 proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via 2a03:b0c0:1:d0::1 dev eth0 proto static metric 1024 pref medium clientqa03 ip route -------------------- default via 159.223.160.1 dev eth0 proto static metric 100 10.10.0.0/16 dev eth0 proto kernel scope link src 10.10.0.5 metric 100 10.116.0.0/20 dev eth1 proto kernel scope link src 10.116.0.2 metric 101 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.6 159.223.160.0/20 dev eth0 proto kernel scope link src 159.223.160.241 metric 100 ::1 dev lo proto kernel metric 256 pref medium 2604:a880:400:d0::/64 dev eth0 proto kernel metric 100 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 1024 pref medium fe80::/64 dev eth1 proto kernel metric 1024 pref medium default via 2604:a880:400:d0::1 dev eth0 proto static metric 100 pref medium clientqa04 ip route -------------------- default via 209.38.224.1 dev eth0 proto static 10.19.0.0/16 dev eth0 proto kernel scope link src 10.19.0.16 10.114.0.0/20 dev eth1 proto kernel scope link src 10.114.0.13 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.16 209.38.224.0/19 dev eth0 proto kernel scope link src 209.38.230.81 ::1 dev lo proto kernel metric 256 pref medium 2a03:b0c0:3:d0::/64 dev eth0 proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via 2a03:b0c0:3:d0::1 dev eth0 proto static metric 1024 pref medium getting initial wg configs for clients clientQA wg -------------------- interface: netmaker public key: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= private key: (hidden) listening port: 51821 peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 37 seconds ago transfer: 2.45 MiB received, 2.45 MiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 40 seconds ago transfer: 1.40 MiB received, 1.26 MiB sent persistent keepalive: every 20 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 42 seconds ago transfer: 2.44 MiB received, 2.43 MiB sent persistent keepalive: every 20 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 10.106.0.10:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 53 seconds ago transfer: 794.27 KiB received, 789.26 KiB sent persistent keepalive: every 20 seconds peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 146.190.147.149:62394 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 1 minute, 12 seconds ago transfer: 2.46 MiB received, 2.46 MiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 1 minute, 14 seconds ago transfer: 2.44 MiB received, 2.45 MiB sent persistent keepalive: every 20 seconds peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 167.172.163.172:62619 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 1 minute, 14 seconds ago transfer: 2.42 MiB received, 2.42 MiB sent persistent keepalive: every 20 seconds peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 1 minute, 47 seconds ago transfer: 2.45 MiB received, 2.44 MiB sent persistent keepalive: every 29 seconds peer: jJAOoN8WLnjAaJ+UaoapMchlzYnuelpkUukom6j40Sg= endpoint: 167.172.163.172:64645 allowed ips: 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128 latest handshake: 1 minute, 56 seconds ago transfer: 2.42 MiB received, 2.41 MiB sent persistent keepalive: every 20 seconds docker-netclient wg interface -------------------- interface: netmaker public key: oKv9uqqhhztUakJ+QnstuA4j2W0/glV5G14JdFhwpDQ= private key: (hidden) listening port: 51821 arm-redhat wg interface -------------------- interface: netmaker public key: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= private key: (hidden) listening port: 51822 peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 165.22.115.179:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 42 seconds ago transfer: 2.45 MiB received, 2.45 MiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 50 seconds ago transfer: 450.18 KiB received, 404.07 KiB sent persistent keepalive: every 20 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 1 minute, 8 seconds ago transfer: 2.46 MiB received, 2.44 MiB sent persistent keepalive: every 20 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 165.22.115.252:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 1 minute, 29 seconds ago transfer: 793.99 KiB received, 787.04 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128, 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128, 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 1 minute, 33 seconds ago transfer: 2.46 MiB received, 2.47 MiB sent persistent keepalive: every 20 seconds peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 2 minutes ago transfer: 2.44 MiB received, 2.42 MiB sent persistent keepalive: every 29 seconds peer: mRi63rqTIKtiZWxJyYSAUpzDjs6AtkRNyDqcVF6FJz4= endpoint: 98.216.56.86:51821 allowed ips: 10.233.54.9/32, fd00:6b13:d07c:4acf::9/128 latest handshake: 2 hours, 43 minutes, 51 seconds ago transfer: 1.26 MiB received, 1.30 MiB sent persistent keepalive: every 20 seconds userspace-wireguard wg interface -------------------- interface: netmaker public key: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= private key: (hidden) listening port: 51821 peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 165.22.115.179:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 50 seconds ago transfer: 2.43 MiB received, 2.44 MiB sent persistent keepalive: every 20 seconds peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 10.124.0.10:51821 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 58 seconds ago transfer: 2.47 MiB received, 2.46 MiB sent persistent keepalive: every 20 seconds peer: jJAOoN8WLnjAaJ+UaoapMchlzYnuelpkUukom6j40Sg= endpoint: 167.172.163.172:64645 allowed ips: 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128 latest handshake: 59 seconds ago transfer: 2.42 MiB received, 2.42 MiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 1 minute, 1 second ago transfer: 1.41 MiB received, 1.27 MiB sent persistent keepalive: every 20 seconds peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 1 minute, 10 seconds ago transfer: 2.44 MiB received, 2.46 MiB sent persistent keepalive: every 20 seconds peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 167.172.163.172:62619 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 1 minute, 15 seconds ago transfer: 2.42 MiB received, 2.41 MiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 1 minute, 19 seconds ago transfer: 2.43 MiB received, 2.44 MiB sent persistent keepalive: every 20 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 165.22.115.252:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 1 minute, 20 seconds ago transfer: 790.49 KiB received, 785.32 KiB sent persistent keepalive: every 20 seconds peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 1 minute, 49 seconds ago transfer: 2.44 MiB received, 2.43 MiB sent persistent keepalive: every 29 seconds clientqa02 wg interface -------------------- interface: netmaker public key: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= private key: (hidden) listening port: 51821 peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 167.172.163.172:62619 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 22 seconds ago transfer: 785.38 KiB received, 781.90 KiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 59 seconds ago transfer: 454.27 KiB received, 409.72 KiB sent persistent keepalive: every 20 seconds peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 10.106.0.9:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 1 minute ago transfer: 789.68 KiB received, 795.05 KiB sent persistent keepalive: every 20 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 1 minute, 20 seconds ago transfer: 785.37 KiB received, 790.63 KiB sent persistent keepalive: every 20 seconds peer: jJAOoN8WLnjAaJ+UaoapMchlzYnuelpkUukom6j40Sg= endpoint: 167.172.163.172:64645 allowed ips: 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128 latest handshake: 1 minute, 25 seconds ago transfer: 781.91 KiB received, 780.08 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 1 minute, 26 seconds ago transfer: 785.71 KiB received, 786.30 KiB sent persistent keepalive: every 20 seconds peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 1 minute, 31 seconds ago transfer: 787.34 KiB received, 794.69 KiB sent persistent keepalive: every 20 seconds peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 1 minute, 35 seconds ago transfer: 788.07 KiB received, 791.47 KiB sent persistent keepalive: every 29 seconds peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 146.190.147.149:62394 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 2 minutes, 4 seconds ago transfer: 783.26 KiB received, 787.07 KiB sent persistent keepalive: every 20 seconds clientqa03 wg interface -------------------- interface: netmaker public key: b/153h32y9fpbajpZg60fvmSUqOSY/kgffXubIkcLAI= private key: (hidden) listening port: 51821 clientqa04 wg interface -------------------- interface: netmaker public key: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= private key: (hidden) listening port: 51825 peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 5 seconds ago transfer: 2.41 MiB received, 2.43 MiB sent persistent keepalive: every 20 seconds peer: jJAOoN8WLnjAaJ+UaoapMchlzYnuelpkUukom6j40Sg= endpoint: 10.114.0.7:51821 allowed ips: 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128 latest handshake: 5 seconds ago transfer: 2.41 MiB received, 2.43 MiB sent persistent keepalive: every 20 seconds peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 146.190.147.149:62394 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 25 seconds ago transfer: 2.47 MiB received, 2.45 MiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 1 minute ago transfer: 1.40 MiB received, 1.26 MiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 1 minute, 37 seconds ago transfer: 2.42 MiB received, 2.43 MiB sent persistent keepalive: every 20 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 165.22.115.252:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 1 minute, 38 seconds ago transfer: 791.65 KiB received, 788.34 KiB sent persistent keepalive: every 20 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 1 minute, 52 seconds ago transfer: 2.42 MiB received, 2.44 MiB sent persistent keepalive: every 20 seconds peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 10.114.0.9:51822 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 1 minute, 56 seconds ago transfer: 2.40 MiB received, 2.42 MiB sent persistent keepalive: every 20 seconds peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 165.22.115.179:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 1 minute, 57 seconds ago transfer: 2.44 MiB received, 2.45 MiB sent persistent keepalive: every 20 seconds peer: mRi63rqTIKtiZWxJyYSAUpzDjs6AtkRNyDqcVF6FJz4= endpoint: 98.216.56.86:51821 allowed ips: 10.233.54.9/32, fd00:6b13:d07c:4acf::9/128 latest handshake: 2 hours, 44 minutes, 4 seconds ago transfer: 1.24 MiB received, 1.30 MiB sent persistent keepalive: every 20 seconds checking initial logs for errors clientQA log errors -------------------- docker-netclient log errors -------------------- arm-redhat log errors -------------------- userspace-wireguard log errors -------------------- clientqa02 log errors -------------------- May 09 14:03:53 clientQA-02 netclient[3288496]: {"time":"2024-05-09T14:03:53.458800647Z","level":"ERROR","source":"wireguard_linux.go 173}","msg":"error removing route","error":"no such process"} May 09 14:03:53 clientQA-02 netclient[3288496]: {"time":"2024-05-09T14:03:53.844316026Z","level":"ERROR","source":"mqhandlers.go 76}","msg":"failed to leave network, please check that local files for network were removed","network":"release-test-241","error":"error(s) leaving nework"} May 09 14:14:32 clientQA-02 netclient[3288496]: {"time":"2024-05-09T14:14:32.693804987Z","level":"ERROR","source":"wireguard_linux.go 197}","msg":"error adding route","error":"invalid argument"} clientqa03 log errors -------------------- clientqa04 log errors -------------------- pulling clients and checking config pulling clientQA -------------------- completed pull for server nm.67-205-143-140.nip.io pulling arm-redhat -------------------- completed pull for server nm.67-205-143-140.nip.io pulling userspace-wireguard -------------------- completed pull for server nm.67-205-143-140.nip.io pulling clientqa02 -------------------- completed pull for server nm.67-205-143-140.nip.io pulling clientqa03 -------------------- completed pull for server nm.67-205-143-140.nip.io pulling clientqa04 -------------------- completed pull for server nm.67-205-143-140.nip.io [netclient] 2024-05-09 15:46:07 [pull.go-80] Pull(): restarting daemon running pings after pull clientQA ping -------------------- nmap found 10.233.54.12 IS DOWN fd00:6b13:d07c:4acf::c IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.11 IS DOWN fd00:6b13:d07c:4acf::b IS UP 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP docker-netclient ping ----------------- nmap found arm-redhat ping ----------------- nmap found 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP userspace-wireguard ping ----------------- nmap found 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP clientqa02 ping ----------------- nmap found 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.12 IS DOWN fd00:6b13:d07c:4acf::c IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP clientqa03 ping ----------------- nmap found clientqa04 ping ----------------- nmap found 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP checking routing after pull clientQA ip route -------------------- default via 165.22.112.1 dev eth0 proto static 10.16.0.0/16 dev eth0 proto kernel scope link src 10.16.0.10 10.106.0.0/20 dev eth1 proto kernel scope link src 10.106.0.9 10.116.0.0/16 dev netmaker src 10.233.54.8 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.8 165.22.112.0/20 dev eth0 proto kernel scope link src 165.22.115.179 ::1 dev lo proto kernel metric 256 pref medium 2a03:b0c0:1:d0::/64 dev eth0 proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via 2a03:b0c0:1:d0::1 dev eth0 proto static metric 1024 pref medium docker-netclient ip route -------------------- default via 170.64.224.1 dev eth0 proto static 10.49.0.0/16 dev eth0 proto kernel scope link src 10.49.0.7 10.126.0.0/20 dev eth1 proto kernel scope link src 10.126.0.2 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.3 170.64.224.0/20 dev eth0 proto kernel scope link src 170.64.238.167 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown ::1 dev lo proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev docker0 proto kernel metric 256 linkdown pref medium arm-redhat ip route -------------------- default via 172.31.16.1 dev eth0 proto dhcp src 172.31.23.197 metric 100 10.116.0.0/16 dev netmaker src 10.233.54.2 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.2 172.31.16.0/20 dev eth0 proto kernel scope link src 172.31.23.197 metric 100 ::1 dev lo proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium userspace-wireguard ip route -------------------- default via 24.144.80.1 dev eth0 proto static 10.48.0.0/16 dev eth0 proto kernel scope link src 10.48.0.14 10.116.0.0/16 dev netmaker src 10.233.54.4 10.124.0.0/20 dev eth1 proto kernel scope link src 10.124.0.13 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.4 24.144.80.0/20 dev eth0 proto kernel scope link src 24.144.84.150 ::1 dev lo proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium clientqa02 ip route -------------------- default via 165.22.112.1 dev eth0 proto static 10.16.0.0/16 dev eth0 proto kernel scope link src 10.16.0.11 10.106.0.0/20 dev eth1 proto kernel scope link src 10.106.0.10 10.116.0.0/16 dev netmaker src 10.233.54.5 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.5 98.216.56.86 via 165.22.112.1 dev eth0 159.223.160.241 via 165.22.112.1 dev eth0 165.22.112.0/20 dev eth0 proto kernel scope link src 165.22.115.252 165.22.115.179 via 165.22.112.1 dev eth0 209.38.230.81 via 165.22.112.1 dev eth0 ::1 dev lo proto kernel metric 256 pref medium 2a03:b0c0:1:d0::/64 dev eth0 proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via 2a03:b0c0:1:d0::1 dev eth0 proto static metric 1024 pref medium clientqa03 ip route -------------------- default via 159.223.160.1 dev eth0 proto static metric 100 10.10.0.0/16 dev eth0 proto kernel scope link src 10.10.0.5 metric 100 10.116.0.0/20 dev eth1 proto kernel scope link src 10.116.0.2 metric 101 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.6 159.223.160.0/20 dev eth0 proto kernel scope link src 159.223.160.241 metric 100 ::1 dev lo proto kernel metric 256 pref medium 2604:a880:400:d0::/64 dev eth0 proto kernel metric 100 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 1024 pref medium fe80::/64 dev eth1 proto kernel metric 1024 pref medium default via 2604:a880:400:d0::1 dev eth0 proto static metric 100 pref medium clientqa04 ip route -------------------- default via 209.38.224.1 dev eth0 proto static 10.19.0.0/16 dev eth0 proto kernel scope link src 10.19.0.16 10.114.0.0/20 dev eth1 proto kernel scope link src 10.114.0.13 10.116.0.0/16 dev netmaker src 10.233.54.16 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.16 209.38.224.0/19 dev eth0 proto kernel scope link src 209.38.230.81 ::1 dev lo proto kernel metric 256 pref medium 2a03:b0c0:3:d0::/64 dev eth0 proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via 2a03:b0c0:3:d0::1 dev eth0 proto static metric 1024 pref medium getting wg configs after pull clientQA wg -------------------- interface: netmaker public key: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= private key: (hidden) listening port: 51821 peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 167.172.163.172:62619 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 6 seconds ago transfer: 36.01 KiB received, 35.84 KiB sent persistent keepalive: every 20 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 10.106.0.10:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 27 seconds ago transfer: 38.79 KiB received, 40.43 KiB sent persistent keepalive: every 20 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 31 seconds ago transfer: 37.28 KiB received, 37.29 KiB sent persistent keepalive: every 20 seconds peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 31 seconds ago transfer: 37.18 KiB received, 36.35 KiB sent persistent keepalive: every 29 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 35 seconds ago transfer: 37.89 KiB received, 34.81 KiB sent persistent keepalive: every 20 seconds peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 36 seconds ago transfer: 34.58 KiB received, 34.41 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 40 seconds ago transfer: 39.60 KiB received, 39.66 KiB sent persistent keepalive: every 20 seconds peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 146.190.147.149:62394 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 1 minute, 52 seconds ago transfer: 37.48 KiB received, 37.87 KiB sent persistent keepalive: every 20 seconds docker-netclient wg interface -------------------- interface: netmaker public key: oKv9uqqhhztUakJ+QnstuA4j2W0/glV5G14JdFhwpDQ= private key: (hidden) listening port: 51821 arm-redhat wg interface -------------------- interface: netmaker public key: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= private key: (hidden) listening port: 51822 peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 31 seconds ago transfer: 40.39 KiB received, 36.86 KiB sent persistent keepalive: every 20 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 165.22.115.252:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 32 seconds ago transfer: 36.45 KiB received, 36.19 KiB sent persistent keepalive: every 20 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 32 seconds ago transfer: 35.97 KiB received, 35.43 KiB sent persistent keepalive: every 20 seconds peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 36 seconds ago transfer: 39.06 KiB received, 38.45 KiB sent persistent keepalive: every 29 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128, 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128, 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 39 seconds ago transfer: 42.63 KiB received, 41.59 KiB sent persistent keepalive: every 20 seconds peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 165.22.115.179:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 41 seconds ago transfer: 34.74 KiB received, 35.45 KiB sent persistent keepalive: every 20 seconds userspace-wireguard wg interface -------------------- interface: netmaker public key: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= private key: (hidden) listening port: 51821 peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 10.124.0.10:51821 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 4 seconds ago transfer: 37.06 KiB received, 36.90 KiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 23 seconds ago transfer: 38.24 KiB received, 34.23 KiB sent persistent keepalive: every 20 seconds peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 167.172.163.172:62619 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 31 seconds ago transfer: 33.96 KiB received, 33.34 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 33 seconds ago transfer: 37.85 KiB received, 38.84 KiB sent persistent keepalive: every 20 seconds peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 34 seconds ago transfer: 35.19 KiB received, 35.97 KiB sent persistent keepalive: every 20 seconds peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 165.22.115.179:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 38 seconds ago transfer: 37.05 KiB received, 37.28 KiB sent persistent keepalive: every 20 seconds peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 38 seconds ago transfer: 38.19 KiB received, 37.82 KiB sent persistent keepalive: every 29 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 165.22.115.252:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 41 seconds ago transfer: 36.61 KiB received, 35.82 KiB sent persistent keepalive: every 20 seconds clientqa02 wg interface -------------------- interface: netmaker public key: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= private key: (hidden) listening port: 51821 peer: jJAOoN8WLnjAaJ+UaoapMchlzYnuelpkUukom6j40Sg= endpoint: 167.172.163.172:64645 allowed ips: 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128 latest handshake: 4 seconds ago transfer: 35.62 KiB received, 34.91 KiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 13 seconds ago transfer: 37.24 KiB received, 33.42 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 29 seconds ago transfer: 35.79 KiB received, 35.91 KiB sent persistent keepalive: every 20 seconds peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 35 seconds ago transfer: 35.96 KiB received, 36.45 KiB sent persistent keepalive: every 20 seconds peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 10.106.0.9:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 35 seconds ago transfer: 38.99 KiB received, 38.02 KiB sent persistent keepalive: every 20 seconds peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 146.190.147.149:62394 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 35 seconds ago transfer: 35.14 KiB received, 34.75 KiB sent persistent keepalive: every 20 seconds peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 37 seconds ago transfer: 37.62 KiB received, 36.43 KiB sent persistent keepalive: every 29 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 43 seconds ago transfer: 35.58 KiB received, 36.61 KiB sent persistent keepalive: every 20 seconds peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 167.172.163.172:62619 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 1 minute, 36 seconds ago transfer: 33.12 KiB received, 34.78 KiB sent persistent keepalive: every 20 seconds clientqa03 wg interface -------------------- interface: netmaker public key: b/153h32y9fpbajpZg60fvmSUqOSY/kgffXubIkcLAI= private key: (hidden) listening port: 51821 clientqa04 wg interface -------------------- interface: netmaker public key: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= private key: (hidden) listening port: 51825 peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 10.114.0.9:51822 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 3 seconds ago transfer: 35.00 KiB received, 36.61 KiB sent persistent keepalive: every 20 seconds peer: jJAOoN8WLnjAaJ+UaoapMchlzYnuelpkUukom6j40Sg= endpoint: 10.114.0.7:51821 allowed ips: 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128 latest handshake: 14 seconds ago transfer: 37.19 KiB received, 37.36 KiB sent persistent keepalive: every 20 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 165.22.115.252:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 39 seconds ago transfer: 36.19 KiB received, 37.62 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 39 seconds ago transfer: 37.29 KiB received, 37.91 KiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 40 seconds ago transfer: 40.16 KiB received, 36.98 KiB sent persistent keepalive: every 20 seconds peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 165.22.115.179:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 41 seconds ago transfer: 38.43 KiB received, 39.36 KiB sent persistent keepalive: every 20 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 41 seconds ago transfer: 37.58 KiB received, 38.31 KiB sent persistent keepalive: every 20 seconds peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 41 seconds ago transfer: 37.75 KiB received, 38.77 KiB sent persistent keepalive: every 20 seconds checking logs for errors after a pull clientQA log errors -------------------- docker-netclient log errors -------------------- arm-redhat log errors -------------------- userspace-wireguard log errors -------------------- clientqa02 log errors -------------------- May 09 14:03:53 clientQA-02 netclient[3288496]: {"time":"2024-05-09T14:03:53.844316026Z","level":"ERROR","source":"mqhandlers.go 76}","msg":"failed to leave network, please check that local files for network were removed","network":"release-test-241","error":"error(s) leaving nework"} May 09 14:14:32 clientQA-02 netclient[3288496]: {"time":"2024-05-09T14:14:32.693804987Z","level":"ERROR","source":"wireguard_linux.go 197}","msg":"error adding route","error":"invalid argument"} clientqa03 log errors -------------------- clientqa04 log errors -------------------- restarting clients and checking config restarting clientQA -------------------- restarting arm-redhat -------------------- restarting userspace-wireguard -------------------- restarting clientqa02 -------------------- restarting clientqa03 -------------------- restarting clientqa04 -------------------- running pings after restart clientQA ping -------------------- nmap found 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP docker-netclient ping ----------------- nmap found arm-redhat ping ----------------- nmap found 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS DOWN 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP userspace-wireguard ping ----------------- nmap found 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP clientqa02 ping ----------------- nmap found 10.233.54.16 IS UP fd00:6b13:d07c:4acf::aa IS UP 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP clientqa03 ping ----------------- nmap found clientqa04 ping ----------------- nmap found 10.233.54.10 IS UP fd00:6b13:d07c:4acf::a IS UP 10.233.54.4 IS UP fd00:6b13:d07c:4acf::4 IS UP 10.233.54.11 IS UP fd00:6b13:d07c:4acf::b IS UP 10.233.54.5 IS UP fd00:6b13:d07c:4acf::5 IS UP 10.233.54.2 IS UP fd00:6b13:d07c:4acf::2 IS UP 10.233.54.8 IS UP fd00:6b13:d07c:4acf::8 IS UP 10.233.54.12 IS UP fd00:6b13:d07c:4acf::c IS UP 10.233.54.1 IS UP fd00:6b13:d07c:4acf::1 IS UP 10.233.54.254 IS UP fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe IS UP 10.233.54.13 IS UP fd00:6b13:d07c:4acf::d IS UP checking routing after restart clientQA ip route -------------------- default via 165.22.112.1 dev eth0 proto static 10.16.0.0/16 dev eth0 proto kernel scope link src 10.16.0.10 10.106.0.0/20 dev eth1 proto kernel scope link src 10.106.0.9 10.116.0.0/16 dev netmaker src 10.233.54.8 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.8 165.22.112.0/20 dev eth0 proto kernel scope link src 165.22.115.179 ::1 dev lo proto kernel metric 256 pref medium 2a03:b0c0:1:d0::/64 dev eth0 proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via 2a03:b0c0:1:d0::1 dev eth0 proto static metric 1024 pref medium docker-netclient ip route -------------------- default via 170.64.224.1 dev eth0 proto static 10.49.0.0/16 dev eth0 proto kernel scope link src 10.49.0.7 10.126.0.0/20 dev eth1 proto kernel scope link src 10.126.0.2 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.3 170.64.224.0/20 dev eth0 proto kernel scope link src 170.64.238.167 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown ::1 dev lo proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev docker0 proto kernel metric 256 linkdown pref medium arm-redhat ip route -------------------- default via 172.31.16.1 dev eth0 proto dhcp src 172.31.23.197 metric 100 10.116.0.0/16 dev netmaker src 10.233.54.2 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.2 172.31.16.0/20 dev eth0 proto kernel scope link src 172.31.23.197 metric 100 ::1 dev lo proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium userspace-wireguard ip route -------------------- default via 24.144.80.1 dev eth0 proto static 10.48.0.0/16 dev eth0 proto kernel scope link src 10.48.0.14 10.116.0.0/16 dev netmaker src 10.233.54.4 10.124.0.0/20 dev eth1 proto kernel scope link src 10.124.0.13 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.4 24.144.80.0/20 dev eth0 proto kernel scope link src 24.144.84.150 ::1 dev lo proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium clientqa02 ip route -------------------- default via 165.22.112.1 dev eth0 proto static 10.16.0.0/16 dev eth0 proto kernel scope link src 10.16.0.11 10.106.0.0/20 dev eth1 proto kernel scope link src 10.106.0.10 10.116.0.0/16 dev netmaker src 10.233.54.5 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.5 98.216.56.86 via 165.22.112.1 dev eth0 159.223.160.241 via 165.22.112.1 dev eth0 165.22.112.0/20 dev eth0 proto kernel scope link src 165.22.115.252 165.22.115.179 via 165.22.112.1 dev eth0 209.38.230.81 via 165.22.112.1 dev eth0 ::1 dev lo proto kernel metric 256 pref medium 2a03:b0c0:1:d0::/64 dev eth0 proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via 2a03:b0c0:1:d0::1 dev eth0 proto static metric 1024 pref medium clientqa03 ip route -------------------- default via 159.223.160.1 dev eth0 proto static metric 100 10.10.0.0/16 dev eth0 proto kernel scope link src 10.10.0.5 metric 100 10.116.0.0/20 dev eth1 proto kernel scope link src 10.116.0.2 metric 101 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.6 159.223.160.0/20 dev eth0 proto kernel scope link src 159.223.160.241 metric 100 ::1 dev lo proto kernel metric 256 pref medium 2604:a880:400:d0::/64 dev eth0 proto kernel metric 100 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 1024 pref medium fe80::/64 dev eth1 proto kernel metric 1024 pref medium default via 2604:a880:400:d0::1 dev eth0 proto static metric 100 pref medium clientqa04 ip route -------------------- default via 209.38.224.1 dev eth0 proto static 10.19.0.0/16 dev eth0 proto kernel scope link src 10.19.0.16 10.114.0.0/20 dev eth1 proto kernel scope link src 10.114.0.13 10.233.54.0/24 dev netmaker proto kernel scope link src 10.233.54.16 209.38.224.0/19 dev eth0 proto kernel scope link src 209.38.230.81 ::1 dev lo proto kernel metric 256 pref medium 2a03:b0c0:3:d0::/64 dev eth0 proto kernel metric 256 pref medium fd00:6b13:d07c:4acf::/64 dev netmaker proto kernel metric 256 pref medium fe80::/64 dev eth1 proto kernel metric 256 pref medium fe80::/64 dev eth0 proto kernel metric 256 pref medium default via 2a03:b0c0:3:d0::1 dev eth0 proto static metric 1024 pref medium getting wg configs after restart clientQA wg -------------------- interface: netmaker public key: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= private key: (hidden) listening port: 51821 peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 146.190.147.149:62394 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 38 seconds ago transfer: 48.22 KiB received, 48.93 KiB sent persistent keepalive: every 20 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 10.106.0.10:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 44 seconds ago transfer: 44.11 KiB received, 45.45 KiB sent persistent keepalive: every 20 seconds peer: jJAOoN8WLnjAaJ+UaoapMchlzYnuelpkUukom6j40Sg= endpoint: 167.172.163.172:64645 allowed ips: 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128 latest handshake: 51 seconds ago transfer: 25.99 KiB received, 25.87 KiB sent persistent keepalive: every 20 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 56 seconds ago transfer: 42.87 KiB received, 45.04 KiB sent persistent keepalive: every 20 seconds peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 1 minute, 27 seconds ago transfer: 47.91 KiB received, 48.89 KiB sent persistent keepalive: every 20 seconds peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 167.172.163.172:62619 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 1 minute, 38 seconds ago transfer: 48.50 KiB received, 47.04 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 1 minute, 43 seconds ago transfer: 49.93 KiB received, 49.27 KiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 1 minute, 44 seconds ago transfer: 51.75 KiB received, 46.47 KiB sent persistent keepalive: every 20 seconds peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 2 minutes, 7 seconds ago transfer: 46.13 KiB received, 47.38 KiB sent persistent keepalive: every 29 seconds docker-netclient wg interface -------------------- interface: netmaker public key: oKv9uqqhhztUakJ+QnstuA4j2W0/glV5G14JdFhwpDQ= private key: (hidden) listening port: 51821 arm-redhat wg interface -------------------- interface: netmaker public key: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= private key: (hidden) listening port: 51822 peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 8 seconds ago transfer: 43.70 KiB received, 43.75 KiB sent persistent keepalive: every 29 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 165.22.115.252:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 52 seconds ago transfer: 42.17 KiB received, 42.66 KiB sent persistent keepalive: every 20 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 1 minute, 9 seconds ago transfer: 45.80 KiB received, 46.45 KiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 1 minute, 27 seconds ago transfer: 49.05 KiB received, 43.94 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128, 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128, 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 1 minute, 29 seconds ago transfer: 52.74 KiB received, 51.20 KiB sent persistent keepalive: every 20 seconds peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 165.22.115.179:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 1 minute, 32 seconds ago transfer: 48.22 KiB received, 47.91 KiB sent persistent keepalive: every 20 seconds userspace-wireguard wg interface -------------------- interface: netmaker public key: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= private key: (hidden) listening port: 51821 peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 17 seconds ago transfer: 42.93 KiB received, 44.16 KiB sent persistent keepalive: every 29 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 165.22.115.252:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 28 seconds ago transfer: 42.27 KiB received, 43.59 KiB sent persistent keepalive: every 20 seconds peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 167.172.163.172:62619 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 37 seconds ago transfer: 44.06 KiB received, 44.09 KiB sent persistent keepalive: every 20 seconds peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 165.22.115.179:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 1 minute, 3 seconds ago transfer: 42.43 KiB received, 41.97 KiB sent persistent keepalive: every 20 seconds peer: jJAOoN8WLnjAaJ+UaoapMchlzYnuelpkUukom6j40Sg= endpoint: 167.172.163.172:64645 allowed ips: 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128 latest handshake: 1 minute, 7 seconds ago transfer: 26.09 KiB received, 25.50 KiB sent persistent keepalive: every 20 seconds peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 1 minute, 10 seconds ago transfer: 45.78 KiB received, 45.80 KiB sent persistent keepalive: every 20 seconds peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 10.124.0.10:51821 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 1 minute, 11 seconds ago transfer: 45.73 KiB received, 44.54 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 1 minute, 12 seconds ago transfer: 43.52 KiB received, 42.39 KiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 1 minute, 12 seconds ago transfer: 48.04 KiB received, 43.50 KiB sent persistent keepalive: every 20 seconds clientqa02 wg interface -------------------- interface: netmaker public key: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= private key: (hidden) listening port: 51821 peer: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= endpoint: 209.38.230.81:51825 allowed ips: 10.233.54.16/32, fd00:6b13:d07c:4acf::aa/128 latest handshake: 10 seconds ago transfer: 39.78 KiB received, 39.54 KiB sent persistent keepalive: every 29 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 29 seconds ago transfer: 42.93 KiB received, 42.27 KiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 46 seconds ago transfer: 44.61 KiB received, 39.78 KiB sent persistent keepalive: every 20 seconds peer: jJAOoN8WLnjAaJ+UaoapMchlzYnuelpkUukom6j40Sg= endpoint: 167.172.163.172:64645 allowed ips: 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128 latest handshake: 47 seconds ago transfer: 40.51 KiB received, 40.31 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 48 seconds ago transfer: 40.60 KiB received, 39.97 KiB sent persistent keepalive: every 20 seconds peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 10.106.0.9:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 52 seconds ago transfer: 41.68 KiB received, 41.83 KiB sent persistent keepalive: every 20 seconds peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 55 seconds ago transfer: 39.63 KiB received, 40.36 KiB sent persistent keepalive: every 20 seconds peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 146.190.147.149:62394 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 55 seconds ago transfer: 40.36 KiB received, 39.91 KiB sent persistent keepalive: every 20 seconds peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 167.172.163.172:62619 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 57 seconds ago transfer: 42.56 KiB received, 41.55 KiB sent persistent keepalive: every 20 seconds clientqa03 wg interface -------------------- interface: netmaker public key: b/153h32y9fpbajpZg60fvmSUqOSY/kgffXubIkcLAI= private key: (hidden) listening port: 51821 clientqa04 wg interface -------------------- interface: netmaker public key: m+6BDZebCE8l/as4Gm6DhrM2Ki/pWIxiKL/WS6JH1TY= private key: (hidden) listening port: 51825 peer: 1kPz0n5yhPiVD8XSdOInXj3otGWo+0jSkPH0/uExqyE= endpoint: 165.22.115.179:51821 allowed ips: 10.233.54.8/32, fd00:6b13:d07c:4acf::8/128 latest handshake: 5 seconds ago transfer: 37.57 KiB received, 37.89 KiB sent persistent keepalive: every 20 seconds peer: l/YZCJFKb0P7K31yAbFWkcynSbDr+htpzemqtM4rzwU= endpoint: 165.22.115.252:51821 allowed ips: 10.233.54.5/32, fd00:6b13:d07c:4acf::5/128 latest handshake: 12 seconds ago transfer: 38.52 KiB received, 38.17 KiB sent persistent keepalive: every 20 seconds peer: 6FTW8sTexxxo2rwnJgaMoW0AB6dFMV2UE3Ou7fd6UWY= endpoint: 54.91.197.18:51822 allowed ips: 10.233.54.2/32, fd00:6b13:d07c:4acf::2/128 latest handshake: 14 seconds ago transfer: 34.97 KiB received, 36.03 KiB sent persistent keepalive: every 20 seconds peer: tbkqg/LAKDrpRBQ7xQ8FiJW7GgfZff37YvK12q8U82A= endpoint: 67.205.143.140:51821 allowed ips: 10.233.54.1/32, fd00:6b13:d07c:4acf::1/128, 10.233.54.254/32, fd00:6b13:d07c:4acf:ffff:ffff:ffff:fffe/128 latest handshake: 17 seconds ago transfer: 36.49 KiB received, 35.65 KiB sent persistent keepalive: every 20 seconds peer: cG7k6wMvemdkZu7SYDBBtvl8Ymiaz/KHvUtXN9Hjyn0= endpoint: 24.144.84.150:51821 allowed ips: 10.233.54.4/32, fd00:6b13:d07c:4acf::4/128 latest handshake: 21 seconds ago transfer: 40.07 KiB received, 39.39 KiB sent persistent keepalive: every 20 seconds peer: xpvnTWhynJxFE2bKcgZcelRWmGJ9A3VJuTPnaazUdF4= endpoint: 98.216.56.86:24410 allowed ips: 10.233.54.10/32, fd00:6b13:d07c:4acf::a/128 latest handshake: 24 seconds ago transfer: 40.10 KiB received, 35.78 KiB sent persistent keepalive: every 20 seconds peer: jJAOoN8WLnjAaJ+UaoapMchlzYnuelpkUukom6j40Sg= endpoint: 10.114.0.7:51821 allowed ips: 10.233.54.11/32, fd00:6b13:d07c:4acf::b/128 latest handshake: 52 seconds ago transfer: 36.39 KiB received, 36.54 KiB sent persistent keepalive: every 20 seconds peer: 8Ga3YaqUWLN6RbiLdEP3U6X/6xmNmM1/dDCp0X4fvF0= endpoint: 10.114.0.9:51822 allowed ips: 10.233.54.12/32, fd00:6b13:d07c:4acf::c/128 latest handshake: 1 minute, 1 second ago transfer: 35.59 KiB received, 35.88 KiB sent persistent keepalive: every 20 seconds peer: v71lEq3DYaWbbEkgBYzS5ADK+Y2/Q+36oxe7PWTiGik= endpoint: 146.190.147.149:62394 allowed ips: 10.233.54.13/32, fd00:6b13:d07c:4acf::d/128 latest handshake: 1 minute, 16 seconds ago transfer: 25.88 KiB received, 24.57 KiB sent persistent keepalive: every 20 seconds checking logs for errors after a restart clientQA log errors -------------------- docker-netclient log errors -------------------- arm-redhat log errors -------------------- userspace-wireguard log errors -------------------- clientqa02 log errors -------------------- clientqa03 log errors -------------------- clientqa04 log errors --------------------