Skip to content

fastpath

Directory actions

More options

Directory actions

More options

Latest commit

 

History

History

fastpath

README.adoc

OONI Fastpath pipeline

blockdiag {
  Collectors -> SSHFeeder -> Ingestion -> ... -> Analysis -> PSQL.events -> RSSFeed
  S3 -> S3Feeder -> Ingestion
  PSQL.events -> Explorer
}

See the makefile file for useful commands.

Run unit tests locally:

make functests

Run devel mode:

make run_devel

Monitor logs:

journalctl -f -t fastpath --utc -o short-precise

# show fields
journalctl -f -t fastpath  -N | sort

# filter by field, e.g.:
journalctl -f -t fastpath --utc CODE_LINE=12

Monitor metrics locally:

make local_monitor_metrics

To set up credentials for development create:

~/ .aws/credentials
~/var/lib/fastpath/ssh/id_ed25519 from /var/lib/fastpath/ssh/id_ed25519 (created by install scripts)
touch /home/federico/var/lib/fastpath/ssh/known_hosts

In development mode, ~ is treated as /

Run development mode:

fastpath --devel --start-day=2019-7-20 --end-day=2019-7-21

On the collectors:

# on fastpath
cat ~/var/lib/fastpath/ssh/id_ed25519.pub
# add pubkey on feeder:
sudo vim /home/sshfeeder/.ssh/authorized_keys

Test connectivity:

cat  /etc/machine-id
ssh [email protected] -i ~/var/lib/fastpath/ssh/id_ed25519