lumen或laravel 将日志上报至kafka

大家都清楚 Lumen 算是一个精简版的 Laravel，针对速度优化了框架的加载方式，多余的功能需要自行开启。那么在精简版中日志的上报配置方面lumen也做了精简，本文中将分别介绍这两者日志上报的基本方法，咱们主要还是依赖hhxsv5/laravel-kafka-logger，下面是方法和步骤（Laravel 和 lumen的安装咱们就略过了）

咱们先安装php的rdkafka扩展

• 要求

  依赖	要求
  php-rdkafka	>=4.0.0

• 安装方法

  git clone --depth 1 https://github.com/edenhill/librdkafka.git /tmp/librdkafka && cd /tmp/librdkafka && ./configure && make -j$(nproc) && make install && rm -rf /tmp/librdkafka
  pecl install rdkafka

安装 laravel-kafka-logger

• 在项目目录,根据您的项目版本运行如下脚本

  # Laravel 5.x
  composer require "hhxsv5/laravel-kafka-logger:~1.0.0"
  # Laravel 6.x & 7.x
  composer require "hhxsv5/laravel-kafka-logger:~2.0.0"

lavael 上配置

1. 修改配置文件 config/logging.php

   return [
     'channels' => [
    //...,  
    'kafka' => Hhxsv5\LKL\KafkaLogger::getDefinition(['topic' => env('LOG_KAFKA_TOPIC', 'laravel-logs')]),    
     ],
   ];

lumen 上配置

1. 在app/Providers目录新增日志扩展类LogServiceProvider.php

   <?php
   namespace App\Providers;
   
   use Hhxsv5\LKL\KafkaLogHandler;
   use Illuminate\Support\ServiceProvider;
   use RdKafka\Conf;
   use Monolog\Logger;
   
   class LogServiceProvider extends ServiceProvider
   {
    /**
     * Configure logging on boot.
     *
     * @return void
     */
    public function boot()
    {
        $this->createLogger([
            'driver'           => 'custom',
            'via'              => static::class,
            'name'             => config('app.name'),
            'topic'            => env('LOG_KAFKA_TOPIC', 'php-logs'),
            'fallback'         => 'daily',
            'flush_timeout_ms' => 5000,
            'settings'         => [ // see https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md
                'metadata.broker.list'    => env('LOG_KAFKA_BROKER_LIST', '127.0.0.1:9092'),
                'queue.buffering.max.ms'  => 3000, // < flush_timeout_ms, send messages ASAP
                'socket.keepalive.enable' => true,
                'log_level'               => LOG_WARNING,
            ],
            // 'tap' => [App\Library\LogFormatter::class], //--格式化日志
        ]);
    }
   
    /**
     * Register the log service.
     *
     * @return void
     */
    public function register()
    {
   
    }
   
    private function createLogger($config){
        if (!isset($config['fallback'])) {
            $config['fallback'] = 'daily';
        }
        if (!isset($config['flush_timeout_ms'])) {
            $config['flush_timeout_ms'] = 60000;
        }
        if (!isset($config['level'])) {
            $config['level'] = Logger::DEBUG;
        }
        if (!isset($config['bubble'])) {
            $config['bubble'] = true;
        }
        $producerConf = new Conf();
        foreach ($config['settings'] as $key => $value) {
            $producerConf->set($key, $value);
        }
        $producerConf->setLogCb(function ($kafka, $level, $facility, $message) use ($config) {
            app('log')->channel($config['fallback'])->info(sprintf('KafkaLogger %s: %s (level: %d)', $facility, $message, $level));
        });
        $producerConf->setErrorCb(function ($kafka, $err, $reason) use ($config) {
            app('log')->channel($config['fallback'])->error(sprintf('KafkaLogger error: %s (reason: %s)', rd_kafka_err2str($err), $reason));
        });
        $producerConf->setDrMsgCb(function ($kafka, $message) use ($config) {
            if ($message->err) {
                app('log')->channel($config['fallback'])->error(sprintf('KafkaLogger delivery fail: %s', $message->errstr()));
            }
        });
        $handler = new KafkaLogHandler($config['name'], $producerConf, $config['topic'], null, $config['fallback'], $config['flush_timeout_ms'], $config['level'], $config['bubble']);
        $this->app['log']->setHandlers([$handler]);
        return $this->app['log'];
    }
   }

2. 注册类到bootstrap/app.php

   $app->register(\App\Providers\LogServiceProvider::class);

最后步骤

1. 修改或新增.env配置文件如下内容

   LOG_CHANNEL=kafka # 注意：这一行lumen不要加的，否则白给
   LOG_KAFKA_BROKER_LIST=kafka:9092 # 多个用英文逗号间隔
   LOG_KAFKA_TOPIC=laravel-logs

2. 测试

   Log::info('测试日志', ['ok-uid-hhxsv5']);

大功告成，祝顺利！