Mikrotik monitoring in Grafana

Had some performance issues with my Mikrotik router. Looking at Tools > Profile, firewall was consuming over 70% CPU

After downgrade to version 6 from 7 and config reset it looks stable.

Wanted to monitor it and found a pretty nice solution https://github.com/akpw/mktxp-stack

A Grafana dashboard https://grafana.com/grafana/dashboards/13679-mikrotik-mktxp-exporter/ exists.

A Mikrotik data exporter via API is available at https://github.com/akpw/mktxp/

docker-compose.yml

services:
  mktxp:
    image: ghcr.io/akpw/mktxp:latest
    container_name: mktxp
    command: --cfg-dir /etc/mktxp export
    volumes:
      - ./mktxp:/etc/mktxp
    ports:
      - "49090:49090"
    restart: unless-stopped

  prometheus:
    image: prom/prometheus:latest
    container_name: prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.retention.time=30d'
    ports:
      - "9090:9090"
    restart: unless-stopped

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    user: "1000"
    volumes:
      - grafana_data:/var/lib/grafana
    ports:
      - "3000:3000"
    restart: unless-stopped

volumes:
  prometheus_data:
  grafana_data:

prometheus.yml

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'mktxp'
    static_configs:
      - targets: ['mktxp:49090']

Make sure API is enabled on Mikrotik and create monitoring user for API access
mktxp.conf

[Mikrotik]
    hostname = 192.168.2.1
    port = 8728
    username = monitor
    password = dut6nez
    use_ssl = False

[default]
    enabled = True
    hostname = localhost
    port = 8728

    username = monitor
    password = password
    credentials_file = ""   # To use an external file in YAML format for both username and password, specify the path here

    custom_labels = None    # Custom labels to be injected to all device metrics, comma-separated key:value (or key=value) pairs

        use_ssl = False                 # enables connection via API-SSL servis
    no_ssl_certificate = False      # enables API_SSL connect without router SSL certificate
    ssl_certificate_verify = False  # turns SSL certificate verification on / off
    ssl_check_hostname = True       # check if the hostname matches the peer cert’s hostname
    ssl_ca_file = ""                # path to the certificate authority file to validate against, leave empty to use system store
    plaintext_login = True          # for legacy RouterOS versions below 6.43 use False

    health = True                   # System Health metrics
    installed_packages = False      # Installed packages
    dhcp = True                     # DHCP general metrics
    dhcp_lease = True               # DHCP lease metrics

    connections = True              # IP connections metrics
    connection_stats = True         # Open IP connections metrics

    interface = True                # Interfaces traffic metrics

    route = True                    # IPv4 Routes metrics
    pool = True                     # IPv4 Pool metrics
    firewall = True                 # IPv4 Firewall rules traffic metrics
    neighbor = True                 # IPv4 Reachable Neighbors
    address_list = None             # Firewall Address List metrics, a comma-separated list of names
    dns = True                      # DNS stats

    ipv6_route = False              # IPv6 Routes metrics
    ipv6_pool = False               # IPv6 Pool metrics
    ipv6_firewall = False           # IPv6 Firewall rules traffic metrics
    ipv6_neighbor = False           # IPv6 Reachable Neighbors
    ipv6_address_list = None        # IPv6 Firewall Address List metrics, a comma-separated list of names

    poe = True                      # POE metrics
    monitor = True                  # Interface monitor metrics
    netwatch = False                 # Netwatch metrics
    public_ip = True                # Public IP metrics
    wireless = True                 # WLAN general metrics
    wireless_clients = True         # WLAN clients metrics
    capsman = False                  # CAPsMAN general metrics
    capsman_clients = False          # CAPsMAN clients metrics

    eoip = False                    # EoIP status metrics
    gre = False                     # GRE status metrics
    ipip = False                    # IPIP status metrics
    lte = False                     # LTE signal and status metrics (requires additional 'test' permission policy on RouterOS v6)
    ipsec = False                   # IPSec active peer metrics
    switch_port = True              # Switch Port metrics

    kid_control_assigned = False    # Allow Kid Control metrics for connected devices with assigned users
    kid_control_dynamic = False     # Allow Kid Control metrics for all connected devices, including those without assigned user

    user = False                     # Active Users metrics
    queue = False                    # Queues metrics

    bfd = False                     # BFD sessions metrics
    bgp = False                     # BGP sessions metrics
    routing_stats = False           # Routing process stats
    certificate = False             # Certificates metrics
    w60g = False

    container = False               # Containers metrics

    remote_dhcp_entry = None        # An MKTXP entry to provide for remote DHCP info / resolution
    remote_capsman_entry = None     # An MKTXP entry to provide for remote capsman info

    use_comments_over_names = True  # when available, forces using comments over the interfaces names
    check_for_updates = False       # check for available ROS updates

Internet speed tracking

Had some issues with internet speed and to provide more info to my ISP I was looking for some internet speed tracking solution.

Found https://github.com/henrywhitaker3/Speedtest-Tracker which uses https://www.speedtest.net/ under the hood.

Speedtest tracker dashboard
services:
  speedtest-tracker:
    container_name: speedtest-tracker
    image: lscr.io/linuxserver/speedtest-tracker:latest
    environment:
      - APP_KEY=base64:wNEbBQtMj7cCO8OVs=
      - AUTH=true
      - DB_CONNECTION=mariadb
      - DB_HOST=db
      - DB_PORT=3306
      - DB_DATABASE=speedtest_tracker
      - DB_USERNAME=root
      - DB_PASSWORD=root
      - OOKLA_EULA_GDPR=true
      - SPEEDTEST_SCHEDULE=0 * * * *
      - TZ=Europe/Bratislava
      - APP_TIMEZONE=Europe/Bratislava
      - DISPLAY_TIMEZONE=Europe/Bratislava
    ports:
      - '8765:80'
    restart: unless-stopped
    volumes:
      - ./config:/config
    depends_on:
      - db

  db:
    image: mariadb:11
    restart: always
    environment:
      - MYSQL_DATABASE=speedtest_tracker
      - MYSQL_USER=root
      - MYSQL_ROOT_PASSWORD=root
    volumes:
      - speedtest-db:/var/lib/mysql
    healthcheck:
      test: ["CMD", "healthcheck.sh", "--connect", "--innodb_initialized"]
      interval: 5s
      retries: 3
      timeout: 5s

volumes:
  speedtest-db:

Found a nice alternative https://github.com/gnmyt/myspeed

Multitail only show errors

Got tired of scanning threw logs for errors in multitail. It would be great to filter them instead me scanning for them.

Fortunately multitail can do that.

Filtering using regular expressions

For filtering MultiTail uses regular expressions. To keep things simple, it uses them the exact same way as ‘grep’ does: ‘-e’ says: a regular expression follows and ‘-v’ says: invert it. Since version 3.5.0 this has changed somewhat: if you want to match the lines that do NOT have a certain pattern, use -ev.

Examples:

multitail -e "gnu-pop3d" /var/log/messages
multitail -v -e "ssh" -v -e "gnu-pop3d" -e "localhost" /var/log/messages

The first example shows only lines from /var/log/messages which have the string “gnu-pop3d” somewhere in them. The second example only shows lines which do not have the string “ssh” and not have the string “gnu-pop3d” and DO HAVE the string “localhost” in them.

For filtering multiple files use -E use regular expression on following files

Now I check docker logs with command:

multitail -cS monolog -E '.*(\[error\]|\.(ERROR|WARNING)| HTTP.* 500 | 404 |\.(ALERT|CRITICAL|EMERG|FATAL)|\[php.*(warn|crit)\]).*' -l 'docker-compose logs --no-color -t -f'

99% of Men Waste Their Lives — CIA Spy Reveals Why

[01:48] a good life is a life spent doing the things you want to do, the things that bring you joy when you have the age and the energy to do them. It makes me super sad whenever I meet people who wait until their 60s, and they retire to be free to try and travel. That’s when they focus on learning the guitar, and that’s when they focus on art, and their body just can’t keep up with them. Their body can’t travel like it used to travel.

[02:26] it’s all about finding joy in the moment today. My son is 12. He plays chess now. He wants to play video games with me now.

From

Shell history hack

Many times I struggle to find the correct shell command in history for my specific need what I used in the past.

So currently I’m adding comment to the end of the command for a specific flow, e.g.

$ sudo pacman -Qdtq | sudo pacman -Rns - # remove orphaned packages

Now I can grep for orphan and found the related command

$ history | grep orphan
14989  pacman -Qqtd # show orphaned
14991  sudo pacman -Qdtq | sudo pacman -Rns - # remove orphaned packages

Remove plus/sticky sign from AwesomeWM window

Was struggling with this and even ChatGPT couldn’t give the right answer to remove the sticky (+) from a window.

Only sticky mention in my rc.lua was awful.titlebar.widget.stickybutton


    local titlebars_enabled = false -- change to true to remove sticky
    if titlebars_enabled and (c.type == "normal" or c.type == "dialog") then
        -- buttons for the titlebar
        local buttons = awful.util.table.join(
                awful.button({ }, 1, function()
                    client.focus = c
                    c:raise()
                    awful.mouse.client.move(c)
                end),
                awful.button({ }, 3, function()
                    client.focus = c
                    c:raise()
                    awful.mouse.client.resize(c)
                end)
                )

        -- Widgets that are aligned to the left
        local left_layout = wibox.layout.fixed.horizontal()
        left_layout:add(awful.titlebar.widget.iconwidget(c))
        left_layout:buttons(buttons)

        -- Widgets that are aligned to the right
        local right_layout = wibox.layout.fixed.horizontal()
        right_layout:add(awful.titlebar.widget.floatingbutton(c))
        right_layout:add(awful.titlebar.widget.maximizedbutton(c))
        right_layout:add(awful.titlebar.widget.stickybutton(c))
        right_layout:add(awful.titlebar.widget.ontopbutton(c))
        right_layout:add(awful.titlebar.widget.closebutton(c))

        -- The title goes in the middle
        local middle_layout = wibox.layout.flex.horizontal()
        local title = awful.titlebar.widget.titlewidget(c)
        title:set_align("center")
        middle_layout:add(title)
        middle_layout:buttons(buttons)

        -- Now bring it all together
        local layout = wibox.layout.align.horizontal()
        layout:set_left(left_layout)
        layout:set_right(right_layout)
        layout:set_middle(middle_layout)

        awful.titlebar(c):set_widget(layout)
    end

Then I spotted it was in an if statement titlebars_enabled that I head false. My solution was temporary replace it to true and restarting AwesomeWM.

Titlebars showed up on all windows

Where I could click on the desired function and toggle it

Design a site like this with WordPress.com
Get started