接口访问统计和数据记录

RequestStatisticsHandler

1. 需求描述

在使用 Tio-Boot 框架时，我们需要实现一个接口访问统计的功能。为此，我们将创建一个 RequestStatisticsHandler 的实现类 DbRequestStatisticsHandler，用于记录并保存每个 HTTP 请求的访问数据。保存数据的逻辑需要自己实现。为问题排查提供依据

2. 实现步骤

2.1 请求 Id

tio-boot 对于请求 分配了两个 Id,分别是 request id 和 channel id,获取方式和值如下

log.info("request id:{}",request.getId());
log.info("channel id:{}",Long.parseLong(request.getChannelContext().getId()));

output

request id:1
channel id:1831315515085565952

我们可以将 channel id 和 request_id 传入到 业务层,业务层在进行日志记录是输出 channel id,request_id

2.2 创建 DbRequestStatisticsHandler 类

首先，我们需要创建一个实现 RequestStatisticsHandler 接口的类 DbRequestStatisticsHandler。在这个类中，我们重写 count 方法，用于处理每个 HTTP 请求并将请求数据保存到数据库。

创建数据表

drop table if exists sys_http_request_statistics;
CREATE TABLE sys_http_request_statistics (
  id BIGINT NOT NULL,
  ip VARCHAR,
  user_id VARCHAR,
  uri VARCHAR,
  header text,
  body text,
  remark VARCHAR (256),
  creator VARCHAR (64) DEFAULT '',
  create_time TIMESTAMP WITHOUT TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
  updater VARCHAR (64) DEFAULT '',
  update_time TIMESTAMP WITHOUT TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
  deleted SMALLINT NOT NULL DEFAULT 0,
  tenant_id BIGINT NOT NULL DEFAULT 0
);

编写业务类,插入数据库

import com.litongjava.db.activerecord.Db;

public class DbRequestStatisticsService {
  public static final String sql = "INSERT INTO sys_http_request_statistics (id, ip, user_id, uri, header, body) values(?,?,?,?,?,?)";

  public void saveDb(long id, String clientIp, Object userId, String requestLine, String header, String body) {
    Db.updateBySql(sql, id, clientIp, userId, requestLine, header, body);
  }
}

实现 RequestStatisticsHandler 的 count 方法,使用异步任务更新数据库

import com.litongjava.jfinal.aop.Aop;
import com.litongjava.open.chat.services.DbRequestStatisticsService;
import com.litongjava.tio.boot.http.handler.RequestStatisticsHandler;
import com.litongjava.tio.http.common.HttpRequest;
import com.litongjava.tio.http.common.utils.HttpIpUtils;
import com.litongjava.tio.utils.thread.TioThreadUtils;

public class DbRequestStatisticsHandler implements RequestStatisticsHandler {

  @Override
  public void count(HttpRequest request) {
    final String requestLine = request.getRequestLine().toString();
    long id = Long.parseLong(request.getChannelContext().getId());
    String authorization = request.getHeader("authorization");
    String token = request.getHeader("token");

    StringBuffer header = new StringBuffer();
    if (authorization != null) {
      header.append("authorization:").append(authorization).append("\n");
    }

    if (token != null) {
      header.append("token:").append(token).append("\n");
    }

    String bodyString = request.getBodyString();

    // 接口访问统计在拦截器之前运行,此时 还有解出id
    Object userId = request.getAttribute("userId");
    String clientIp = HttpIpUtils.getRealIp(request);

    // 使用ExecutorService异步执行任务
    TioThreadUtils.submit(() -> {
      try {
        Aop.get(DbRequestStatisticsService.class).saveDb(id, clientIp, userId, requestLine, header.toString(), bodyString);
      } catch (Exception e) {
        e.printStackTrace();
      }
    });
  }
}

{
  "id": 1831320535364435968,
  "ip": "0:0:0:0:0:0:0:1",
  "user_id": null,
  "uri": "GET /playwright/id HTTP/1.1",
  "header": "",
  "body": null,
  "remark": null,
  "creator": "",
  "create_time": "2024-09-04T03:18:37.567767",
  "updater": "",
  "update_time": "2024-09-04T03:18:37.567767",
  "deleted": 0,
  "tenant_id": 0
}

2.3 配置 TioBootServer

接下来，我们需要编写 TioBootServerConfig 类，将 DbRequestStatisticsHandler 添加到 TioBootServer 中。

import com.litongjava.tio.boot.server.TioBootServer;

public class TioBootServerConfig {

  public void config() {
    // 创建 DbRequestStatisticsHandler 实例
    DbRequestStatisticsHandler dbRequestStatisticsHandler = new DbRequestStatisticsHandler();

    // 获取 TioBootServer 实例
    TioBootServer server = TioBootServer.me();

    // 将 DbRequestStatisticsHandler 设置到 TioBootServer
    server.setRequestStatisticsHandler(dbRequestStatisticsHandler);
  }
}

3. 效果说明

在 Tio-Boot Server 收到请求后，会自动执行 RequestStatisticsHandler 的 count 方法。在我们实现的 DbRequestStatisticsHandler 类中，这个方法会记录请求的 URI 并调用自定义的 方法将数据保存到数据库。

通过以上步骤，我们实现了一个简单的接口访问统计功能，并且可以根据需求进一步完善数据保存的逻辑。

ResponseStatisticsHandler

1. 需求描述

在使用 Tio-Boot 框架时，我们需要实现一个接口访问统计的功能,统计接口的耗时间。为此，我们将创建一个 ResponseStatisticsHandler 的实现类 MyHttpResponseStatisticsHandler，用于记录并保存每个 HTTP 请求的访问时间到数据库。保存数据的逻辑需要自己实现。

2. 创建表

CREATE TABLE sys_http_response_statistics (
  id BIGINT NOT NULL,
  ip VARCHAR,
  user_id VARCHAR,
  method VARCHAR(10),
  uri VARCHAR(256),
  elapsed BIGINT,
  remark VARCHAR (256),
  creator VARCHAR (64) DEFAULT '',
  create_time TIMESTAMP WITHOUT TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
  updater VARCHAR (64) DEFAULT '',
  update_time TIMESTAMP WITHOUT TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
  deleted SMALLINT NOT NULL DEFAULT 0,
  tenant_id BIGINT NOT NULL DEFAULT 0
);

3.保存接口数据到数据库

MyHttpResponseStatisticsService

import com.litongjava.db.activerecord.Db;
import com.litongjava.db.activerecord.Record;
import com.litongjava.tio.http.common.HttpMethod;
import com.litongjava.tio.utils.snowflake.SnowflakeIdUtils;

public class MyHttpResponseStatisticsService {

  public void updateDb(HttpMethod method, String uri, Object userId, String clientIp, long elapsed) {
    Record record = new Record();
    record.set("id", SnowflakeIdUtils.id());
    record.set("ip", clientIp);
    record.set("user_id", userId);
    record.set("method", method.toString());
    record.set("uri", uri);
    record.set("elapsed", elapsed);

    try {
      Db.save("sys_http_response_statistics", record);
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

4. 收集访问数据

MyHttpResponseStatisticsHandler

import com.litongjava.jfinal.aop.Aop;
import com.litongjava.tio.boot.http.handler.ResponseStatisticsHandler;
import com.litongjava.tio.http.common.HttpMethod;
import com.litongjava.tio.http.common.HttpRequest;
import com.litongjava.tio.http.common.HttpResponse;
import com.litongjava.tio.http.common.RequestLine;
import com.litongjava.tio.http.common.utils.HttpIpUtils;
import com.litongjava.tio.utils.thread.TioThreadUtils;

public class MyHttpResponseStatisticsHandler implements ResponseStatisticsHandler {

  @Override
  public void count(HttpRequest request, RequestLine requestLine, HttpResponse httpResponse, Object userId,
      long elapsed) {
    HttpMethod method = requestLine.getMethod();

    String requestURI = requestLine.getPath();
    String clientIp = HttpIpUtils.getRealIp(request);

    // 异步保存
    TioThreadUtils.getFixedThreadPool().submit(() -> {
      Aop.get(MyHttpResponseStatisticsService.class).updateDb(method, requestURI, userId, clientIp, elapsed);
    });
  }
}

5.配置 MyHttpResponseStatisticsHandler

TioBootServerConfig

import com.litongjava.jfinal.aop.annotation.AConfiguration;
import com.litongjava.jfinal.aop.annotation.AInitialization;
import com.litongjava.tio.boot.server.TioBootServer;

@AConfiguration
public class TioBootServerConfig {

  @Initialization
  public void config() {
    TioBootServer server = TioBootServer.me();
    MyHttpResponseStatisticsHandler dbResponseStatisticsHandler = new MyHttpResponseStatisticsHandler();
    server.setResponseStatisticsHandler(dbResponseStatisticsHandler);
  }
}

6.显示结果

sys_http_response_statistics

id	ip	user_id	method	uri	elapsed	remark	creator	create_time	updater	update_time	deleted	tenant_id
392890770514042880	0:0:0:0:0:0:0:1	nil	GET	/	16			18/6/2024 12:08:37.807929		18/6/2024 12:08:37.807929	0	0
392890861243531264	0:0:0:0:0:0:0:1	nil	GET	/	0			18/6/2024 12:08:59.439744		18/6/2024 12:08:59.439744	0	0

记录请求和响应数据

面是完整的实现思路和代码。我们将创建一个新的表来记录请求和响应数据，并在处理响应时筛选出只记录 JSON 类型的响应。

1. 创建新的数据表

这张表会同时存储请求和响应的相关数据。

drop table if exists sys_http_request_response_statistics;
CREATE TABLE sys_http_request_response_statistics (
  id BIGINT NOT NULL primary key,
  channel_id BIGINT NOT NULL,
  request_id BIGINT NOT NULL,
  request_ip VARCHAR,
  request_uri VARCHAR(256),
  request_header TEXT,
  request_content_type VARCHAR(128),
  request_body TEXT,
  response_status_code INT,
  response_body TEXT,
  elapsed BIGINT,
  update_time TIMESTAMP WITHOUT TIME ZONE NOT NULL DEFAULT CURRENT_TIMESTAMP,
  deleted SMALLINT NOT NULL DEFAULT 0,
  tenant_id BIGINT NOT NULL DEFAULT 0
);

2. 创建业务逻辑类

为了实现数据的保存逻辑，我们将扩展现有的 DbRequestStatisticsService 和 MyHttpResponseStatisticsService，并合并请求和响应数据的保存。

import com.litongjava.db.activerecord.Db;
import com.litongjava.db.activerecord.Record;
import com.litongjava.tio.utils.mcid.McIdUtils;

public class RequestResponseStatisticsService {
  public void save(long channelId, long requestId, String requestIp, String requestUri, String requestHeader, String contentType, String requestBody, int responseStatusCode, String responseBody,
      long elapsed) {

    Record record = new Record();
    record.set("id", McIdUtils.id());
    record.set("channel_id", channelId);
    record.set("request_id", requestId);
    record.set("request_ip", requestIp);
    record.set("request_uri", requestUri);
    record.set("request_header", requestHeader);
    record.set("request_content_type", contentType);
    record.set("request_body", requestBody);
    record.set("response_status_code", responseStatusCode);
    record.set("response_body", responseBody);
    record.set("elapsed", elapsed);

    try {
      Db.save("sys_http_request_response_statistics", record);
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

3. 扩展 ResponseStatisticsHandler 实现

接下来，我们需要在 ResponseStatisticsHandler 中记录响应数据，并筛选出只有 Content-Type 为 JSON 的响应。

import java.util.concurrent.CompletableFuture;
import com.litongjava.jfinal.aop.Aop;
import com.litongjava.open.chat.services.RequestResponseStatisticsService;
import com.litongjava.tio.boot.http.handler.ResponseStatisticsHandler;
import com.litongjava.tio.http.common.HeaderValue;
import com.litongjava.tio.http.common.HttpRequest;
import com.litongjava.tio.http.common.HttpResponse;
import com.litongjava.tio.http.common.RequestLine;
import com.litongjava.tio.http.common.utils.HttpIpUtils;

import lombok.extern.slf4j.Slf4j;

@Slf4j
public class MyRequestResponseStatisticsHandler implements ResponseStatisticsHandler {

  RequestResponseStatisticsService service = Aop.get(RequestResponseStatisticsService.class);

  @Override
  public void count(HttpRequest request, RequestLine requestLine, HttpResponse httpResponse, Object userId, long elapsed) {

    String path = requestLine.getPath();
    if ("/v1/embeddings".equals(path)) {
      log.info("skip /v1/embeddings");
      return;
    }

    String clientIp = HttpIpUtils.getRealIp(request);
    String requestUri = requestLine.toString();
    long id = Long.parseLong(request.getChannelContext().getId());

    String contentType = request.getContentType();

    String authorization = request.getHeader("authorization");
    String token = request.getHeader("token");
    StringBuffer header = new StringBuffer();
    if (authorization != null) {
      header.append("authorization:").append(authorization).append("\n");
    }

    if (token != null) {
      header.append("token:").append(token).append("\n");
    }

    String requestBody = request.getBodyString();

    int responseStatusCode = httpResponse.getStatus().getStatus();
    HeaderValue contentTypeHeader = httpResponse.getContentType();

    // 异步保存
    CompletableFuture.runAsync(() -> {
      String responseContentType = null;
      String responseBody = null;
      if (contentTypeHeader != null) {
        responseContentType = contentTypeHeader.toString();
        if (responseContentType.contains("application/json")) {
          byte[] body = httpResponse.getBody();
          if (body != null && body.length > 0) {
            responseBody = new String(body);
          }
          service.save(id, clientIp, requestUri, header.toString(), contentType, requestBody, responseStatusCode, responseBody, elapsed);
        }
      }
    });
  }
}

4. 配置 TioBootServer

将我们新的 MyRequestResponseStatisticsHandler 配置到 TioBootServer 中。

import com.litongjava.jfinal.aop.annotation.AConfiguration;
import com.litongjava.jfinal.aop.annotation.AInitialization;
import com.litongjava.tio.boot.server.TioBootServer;

@AConfiguration
public class TioBootServerConfig {

  @Initialization
  public void config() {
    TioBootServer server = TioBootServer.me();
    server.setResponseStatisticsHandler(new MyRequestResponseStatisticsHandler());
  }
}

5. 效果

sys_http_request_response_statistics

{
  "id": 7070054257091008,
  "channel_id": 1833969868340445184,
  "request_id": 4,
  "request_ip": "0:0:0:0:0:0:0:1",
  "request_uri": "GET /api/v1/chat/create?name=hi&school_id=2 HTTP/1.1",
  "request_header": "authorization:Bearer PAfvyWa268y0on8MbxHhfYY21rvi9Sx8\n",
  "request_content_type": null,
  "request_body": null,
  "response_status_code": 200,
  "response_body": "{\"code\":1,\"data\":{\"name\":\"hi\",\"id\":\"f96bb45f-8b0d-46fe-9918-279f12f4c60e\"},\"ok\":true}",
  "elapsed": 218,
  "update_time": "2024-09-04T04:00:53.25183",
  "deleted": 0,
  "tenant_id": 0
}

通过以上步骤，我们实现了完整的请求和响应统计功能，并确保只记录 Content-Type 为 JSON 的响应数据。