通过本文章,可以完成多级缓存架构中的Lua缓存。
一、nginx服务
在docker/docker-compose.yml
中添加nginx服务块。
nginx:
container_name: nginx
image: nginx:stable
volumes:
- ./nginx/conf/nginx.conf:/etc/nginx/nginx.conf
- ./nginx/conf/conf.d/default.conf:/etc/nginx/conf.d/default.conf
- ./nginx/dist:/usr/share/nginx/dist
ports:
- "8080:8080"
networks:
multi-cache:
ipv4_address: 172.30.3.3
删除原来docker里的multiCache
项目并停止springboot
应用。
nginx
部分配置如下,监听端口为8080
,并且将请求反向代理至172.30.3.11
,下一小节,将openresty
固定在172.30.3.11
。
upstream nginx-cluster {
server 172.30.3.11;
}
server {
listen 8080;
listen [::]:8080;
server_name localhost;
location /api {
proxy_pass http://nginx-cluster;
}
}
重新启动multiCache
看看nginx前端网页效果。
docker-compose -p multi-cache up -d
访问http://localhost:8080/item.html?id=10001
查询id=10001商品页
这里是假数据,前端页面会向/api/item/10001
发送数据请求。
二、OpenResty服务
1. 服务块定义
在docker/docker-compose.yml
中添加openresty1
服务块。
openresty1:
container_name: openresty1
image: openresty/openresty:1.21.4.3-3-jammy-amd64
volumes:
- ./openresty1/conf/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf
- ./openresty1/conf/conf.d/default.conf:/etc/nginx/conf.d/default.conf
- ./openresty1/lua:/usr/local/openresty/nginx/lua
- ./openresty1/lualib/common.lua:/usr/local/openresty/lualib/common.lua
networks:
multi-cache:
ipv4_address: 172.30.3.11
2. 配置修改
前端向后端发送/api/item/10001
请求关于id=10001商品信息。
根据nginx
的配置内容,这个请求首先被nginx
拦截,反向代理到172.30.3.11
(即openresty1
)。
upstream nginx-cluster {
server 172.30.3.11;
}
server {
location /api {
proxy_pass http://nginx-cluster;
}
}
openresty1
收到的也是/api/item/10001
,同时,openresty
将/api/item/(\d+)
请求代理到指定lua
程序,在lua
程序中完成数据缓存。
因此,openresty
的conf/conf.d/default.conf
如下
upstream tomcat-cluster {
hash $request_uri;
server 172.30.3.4:8081;
# server 172.30.3.5:8081;
}
server {
listen 80;
listen [::]:80;
server_name localhost;
# intercept /item and join lua
location ~ /api/item/(\d+) {
default_type application/json;
content_by_lua_file lua/item.lua;
}
# intercept lua and redirect to back-end
location /path/ {
rewrite ^/path/(.*)$ /$1 break;
proxy_pass http://tomcat-cluster;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/dist;
}
}
conf/nginx.conf
在http
块最后添加3行,引入依赖。
#lua 模块
lua_package_path "/usr/local/openresty/lualib/?.lua;;";
#c模块
lua_package_cpath "/usr/local/openresty/lualib/?.so;;";
#本地缓存
lua_shared_dict item_cache 150m;
3. Lua程序编写
common.lua
被挂载到lualib
,表示可以被其他lua当做库使用,内容如下
-- 创建一个本地缓存对象item_cache
local item_cache = ngx.shared.item_cache;
-- 函数,向openresty本身发送类似/path/item/10001请求,根据conf配置,将被删除/path前缀并代理至tomcat程序
local function read_get(path, params)
local rsp = ngx.location.capture('/path'..path,{
method = ngx.HTTP_GET,
args = params,
})
if not rsp then
ngx.log(ngx.ERR, "http not found, path: ", path, ", args: ", params);
ngx.exit(404)
end
return rsp.body
end
-- 函数,如果本地有缓存,使用缓存,如果没有代理到tomcat然后将数据存入缓存
local function read_data(key, expire, path, params)
-- query local cache
local rsp = item_cache:get(key)
-- query tomcat
if not rsp then
ngx.log(ngx.ERR, "redis cache miss, try tomcat, key: ", key)
rsp = read_get(path, params)
end
-- write into local cache
item_cache:set(key, rsp, expire)
return rsp
end
local _M = {
read_data = read_data
}
return _M
item.lua
是处理来自形如/api/item/10001
请求的程序,内容如下
-- include
local commonUtils = require('common')
local cjson = require("cjson")
-- get url params 10001
local id = ngx.var[1]
-- redirect item, 缓存过期时间1800s, 适合长时间不改变的数据
local itemJson = commonUtils.read_data("item:id:"..id, 1800,"/item/"..id,nil)
-- redirect item/stock, 缓存过期时间4s, 适合经常改变的数据
local stockJson = commonUtils.read_data("item:stock:id:"..id, 4 ,"/item/stock/"..id, nil)
-- json2table
local item = cjson.decode(itemJson)
local stock = cjson.decode(stockJson)
-- combine item and stock
item.stock = stock.stock
item.sold = stock.sold
-- return result
ngx.say(cjson.encode(item))
4. 总结
- 这里
lua
对item
(tb_item表)和stock
(tb_stock表)两个信息都有缓存,并使用cjson
库将两者合并后返回到前端。 - 关于
expire
时效性的问题,如果后台改变了数据,但是openresty
关于此数据的缓存未过期,前端得到的是旧数据
。 - 大致来说openresty = nginx + lua,不仅具有
nginx
反向代理的能力,还能介入lua
程序进行扩展。
三、运行
到此为止,docker-compose.yml应该如下
version: '3.8'
networks:
multi-cache:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.30.3.0/24
services:
mysql:
container_name: mysql
image: mysql:8
volumes:
- ./mysql/conf/my.cnf:/etc/mysql/conf.d/my.cnf
- ./mysql/data:/var/lib/mysql
- ./mysql/logs:/logs
ports:
- "3306:3306"
environment:
- MYSQL_ROOT_PASSWORD=1009
networks:
multi-cache:
ipv4_address: 172.30.3.2
nginx:
container_name: nginx
image: nginx:stable
volumes:
- ./nginx/conf/nginx.conf:/etc/nginx/nginx.conf
- ./nginx/conf/conf.d/default.conf:/etc/nginx/conf.d/default.conf
- ./nginx/dist:/usr/share/nginx/dist
ports:
- "8080:8080"
networks:
multi-cache:
ipv4_address: 172.30.3.3
openresty1:
container_name: openresty1
image: openresty/openresty:1.21.4.3-3-jammy-amd64
volumes:
- ./openresty1/conf/nginx.conf:/usr/local/openresty/nginx/conf/nginx.conf
- ./openresty1/conf/conf.d/default.conf:/etc/nginx/conf.d/default.conf
- ./openresty1/lua:/usr/local/openresty/nginx/lua
- ./openresty1/lualib/common.lua:/usr/local/openresty/lualib/common.lua
networks:
multi-cache:
ipv4_address: 172.30.3.11
启动各项服务
docker-compose -p multi-cache up -d
启动springboot
程序。
四、测试
清空openresty
容器日志。
访问http://localhost:8080/item.html?id=10001
查看openresty
容器日志,可以看到两次commonUtils.read_data
都没有缓存,于是代理到tomcat
,可以看到springboot
日志出现查询相关记录。
2024-01-12 11:45:53 2024/01/12 03:45:53 [error] 7#7: *1 [lua] common.lua:99: read_data(): redis cache miss, try tomcat, key: item:id:10001, client: 172.30.3.3, server: localhost, request: "GET /api/item/10001 HTTP/1.0", host: "nginx-cluster", referrer: "http://localhost:8080/item.html?id=10001"
2024-01-12 11:45:53 2024/01/12 03:45:53 [error] 7#7: *1 [lua] common.lua:99: read_data(): redis cache miss, try tomcat, key: item:stock:id:10001 while sending to client, client: 172.30.3.3, server: localhost, request: "GET /api/item/10001 HTTP/1.0", host: "nginx-cluster", referrer: "http://localhost:8080/item.html?id=10001"
2024-01-12 11:45:53 172.30.3.3 - - [12/Jan/2024:03:45:53 +0000] "GET /api/item/10001 HTTP/1.0" 200 486 "http://localhost:8080/item.html?id=10001" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0"
再次访问此网址,强制刷新+禁用浏览器缓存+更换浏览器
间隔超过4s但小于1800s时,日志如下,只出现一次miss。
2024-01-12 11:48:04 2024/01/12 03:48:04 [error] 7#7: *4 [lua] common.lua:99: read_data(): redis cache miss, try tomcat, key: item:stock:id:10001, client: 172.30.3.3, server: localhost, request: "GET /api/item/10001 HTTP/1.0", host: "nginx-cluster", referrer: "http://localhost:8080/item.html?id=10001"
2024-01-12 11:48:04 172.30.3.3 - - [12/Jan/2024:03:48:04 +0000] "GET /api/item/10001 HTTP/1.0" 200 486 "http://localhost:8080/item.html?id=10001" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0"
再次访问此网址,强制刷新+禁用浏览器缓存+更换浏览器
间隔小于4s,日志如下,未出现miss。
2024-01-12 11:49:16 172.30.3.3 - - [12/Jan/2024:03:49:16 +0000] "GET /api/item/10001 HTTP/1.0" 200 486 "http://localhost:8080/item.html?id=10001" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36 Edg/121.0.0.0"
五、高可用集群
1. openresty
对于openresty
高可用,可以部署多个openresty docker
实例,并在nginx
的docker/nginx/conf/conf.d/default.conf
的upstream nginx-cluster
将多个openresty地址添加进去即可。比如
upstream nginx-cluster {
hash $request_uri;
# hash $request_uri consistent;
server 172.30.3.11;
server 172.30.3.12;
server 172.30.3.13;
}
多个openresty
无论是conf还是lua都保持一致即可。
并且使用hash $request_uri
负载均衡作为反向代理策略,防止同一请求被多个实例缓存数据。文章来源:https://www.toymoban.com/news/detail-793413.html
2. tomcat
对于springboot
程序高可用,也是类似。可以部署多个springboot docker
实例,并在openresty
的docker/openresty1/conf/conf.d/default.conf
的upstream nginx-cluster
将多个springboot地址添加进去即可。比如文章来源地址https://www.toymoban.com/news/detail-793413.html
upstream tomcat-cluster {
hash $request_uri;
server 172.30.3.4:8081;
server 172.30.3.5:8081;
}
到了这里,关于多级缓存架构(三)OpenResty Lua缓存的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!