简介
今天去定位一个nodeport的问题,发现curl 127.0.0.1:32000 访问nodeport的时候会规律的hang住,本来以为是后端服务的问题,但是curl管理ip:nodeport 是正常的。这个就奇怪了,深入研究了下发现 ipvs模式下是不支持这样访问的,如果想使用 localhost: 得使用iptables模式。
下面是一个歪果仁的解释
After a little dig into the kernel, failing to connect localhost: can be explained.
Assume we are visiting http://127.0.0.1:, every packet will first pass through ip_vs_nat_xmit. Then after some check, it runs to https://github.com/torvalds/linux/blob/v4.18/net/netfilter/ipvs/ip_vs_xmit.c#L756. __ip_vs_get_out_rt is used to search for route to remote server, k8s pods in our case. Take a deeper look at __ip_vs_get_out_rt, after validating route cache or finding route via do_output_route4, it comes to crosses_local_route_boundary to judge whether the searched route can pass cross-local-route-boundary check.
Copy the code of crosses_local_route_boundary here and go deeper.
static inline bool crosses_local_route_boundary(int skb_af, struct sk_buff *skb,
int rt_mode,
bool new_rt_is_local)
{
bool rt_mode_allow_local = !!(rt_mode & IP_VS_RT_MODE_LOCAL);
bool rt_mode_allow_non_local = !!(rt_mode & IP_VS_RT_MODE_NON_LOCAL);
bool rt_mode_allow_redirect = !!(rt_mode & IP_VS_RT_MODE_RDR);
bool source_is_loopback;
bool old_rt_is_local;
#ifdef CONFIG_IP_VS_IPV6
/* omit ipv6 */
#endif
{
source_is_loopback = ipv4_is_loopback(ip_hdr(skb)->saddr);
old_rt_is_local = skb_rtable(skb)->rt_flags & RTCF_LOCAL;
}
if (unlikely(new_rt_is_local)) {
if (!rt_mode_allow_local)
return true;
if (!rt_mode_allow_redirect && !old_rt_is_local)
return true;
} else {
if (!rt_mode_allow_non_local)
return true;
if (source_is_loopback)
return true;
}
return false;
}
For nat mode of IPVS, rt_mode is assigned as IP_VS_RT_MODE_LOCAL | IP_VS_RT_MODE_NON_LOCAL | IP_VS_RT_MODE_RDR as https://github.com/torvalds/linux/blob/v4.18/net/netfilter/ipvs/ip_vs_xmit.c#L757-L759 indicates and new_rt_is_local is 0 due to https://github.com/torvalds/linux/blob/v4.18/net/netfilter/ipvs/ip_vs_xmit.c#L363.
source_is_loopback will be true because source address ip_hdr(skb)->saddr is 127.0.0.1. The five booleans defined in crosses_local_route_boundary will all be true in this case. So we finally fall into here
if (source_is_loopback)
return true;
and never pass the cross-local-route-boundary check.
解决方案
目前看如果想这样使用1是修改内核,这个可能比较难实现,另一个方案是修改路由表,修改源ip的地址,也是一种曲线救国的方式文章来源:https://www.toymoban.com/news/detail-837225.html
sudo ip route change 127.0.0.1 dev lo proto kernel scope host src <node ip> table local
引用
https://github.com/kubernetes/kubernetes/issues/96879
https://github.com/kubernetes/kubernetes/issues/67730
https://serverfault.com/questions/851221/what-is-the-local-routing-table-used-for文章来源地址https://www.toymoban.com/news/detail-837225.html
到了这里,关于k8s ipvs 模式下不支持 localhost:<nodeport>方式访问服务的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!