{"config":{"lang":["en","de"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"index.html","title":"Home","text":"
__ __ _\n\\ \\ / /_ _ _ __ (_) ___\n \\ V / _` | '_ \\| |/ __|\n | | (_| | | | | | (__\n |_|\\__,_|_| |_|_|\\___|\nYet another node info collector\n
yanic
is a respondd client that fetches, stores and publishes information about a Freifunk network.
A little overview of yanic in connection with other software:
"},{"location":"about.html#how-respondd-works","title":"How respondd works","text":"It sends the gluon-neighbour-info
request and collects the answers.
It will send UDP packets with multicast address ff05:0:0:0:0:0:2:1001
and port 1001
.
If a node does not answer, it will request with the last know address under the port 1001
.
VPNs (respondd for servers):
Nodes (respondd for nodes): gluon
"},{"location":"about.html#alternative-collectors-of-respondd-data","title":"Alternative collectors of respondd data:","text":"Databases:
Visualization from Databases: Grafana
Output:
A little project starts, to collect meshviewer-ffrgb data from multiple communities.
List of communities: https://github.com/genofire/meshviewer-collector
Meshviewer with colleced data: multi.meshviewer.org Status: data.meshviewer.org/collector.
"},{"location":"dev/database.html","title":"Add new database type","text":"Write a new package to implement the interface database.Connection:
type Connection interface {\n InsertNode(node *runtime.Node)\n\n InsertLink(*runtime.Link, time.Time)\n\n InsertGlobals(*runtime.GlobalStats, time.Time, string)\n\n PruneNodes(deleteAfter time.Duration)\n\n Close()\n}\n
InsertNode is stores statistics per node
InsertLink is stores statistics per link
InsertGlobals is stores global statistics (by site_code
, and \"global\" like in runtime.GLOBAL_SITE
overall sites).
PruneNodes is prunes historical per-node data
Close is called during shutdown of Yanic.
For startup, you need to bind your database type by calling database.RegisterAdapter(\"typeofdatabase\",ConnectFunction)
it should be in the func init() {}
of your package.
The typeofdatabase is used as mapping in the configuration [[database.connection.typeofdatabase]]
the map[string]interface{}
of the content are parsed to the ConnectFunction and on of your implemented Connection
or a error
is needed as result.
Short: the function signature of ConnectFunction should be func Connect(configuration interface{}) (Connection, error)
At last add you import string to compile the your database as well in this all package.
TIP: take a look in the easy database type logging.
"},{"location":"dev/output.html","title":"Add new output type","text":"Write a new package to implement the interface output.Output:
type Output interface {\n Save(nodes *runtime.Nodes)\n}\n
Save a pre-filtered state of the Nodes
For startup, you need to bind your output type by calling output.RegisterAdapter(\"typeofoutput\",Register)
it should be in the func init() {}
of your package.
The typeofoutput is used as mapping in the configuration [[nodes.output.typeofoutput]]
the map[string]interface{}
of the content are parsed to the Register and on of your implemented Output
or a error
is needed as result.
Short: the function signature of Register should be func Register(configuration map[string]interface{}) (Output, error)
At last add you import string to compile the your database as well in this all package.
"},{"location":"docs/configuration.html","title":"Configuration","text":"Here you would find a long description, maybe the description in example file are enough for you.
The config file for Yanic written in \"Tom's Obvious, Minimal Language.\" syntax. (if you need somethink multiple times, checkout out the [[array of table]] section)
"},{"location":"docs/configuration.html#respondd","title":"[respondd]","text":"Group for configuration of respondd request.
# Send respondd request to update information\n[respondd]\nenable = true # (1)\n# Delay startup until a multiple of the period since zero time\nsynchronize = \"1m\" # (2)\n# how often request per multicast\ncollect_interval = \"1m\" # (3)\n\n# If you have custom respondd fields, you can ask Yanic to also collect these.\n#[[respondd.custom_field]] (4)\n#name = zip\n# You can use arbitrary GJSON expressions here, see https://github.com/tidwall/gjson\n# We expect this expression to return a string.\n#path = nodeinfo.location.zip\n\n# table of a site to save stats for (not exists for global only)\n#[respondd.sites.example] (5)\n## list of domains on this site to save stats for (empty for global only)\n#domains = [\"city\"] (6)\n\n\n# interface that has an IP in your mesh network\n[[respondd.interfaces]] # (7)\n# name of interface on which this collector is running\nifname = \"br-ffhb\" # (8) \n# ip address which is used for sending\n# (optional - without definition used a address of ifname - preferred link local)\nip_address = \"fd2f:5119:f2d::5\" # (9)\n# disable sending multicast respondd request\n# (for receiving only respondd packages e.g. database respondd)\n#send_no_request = false (10)\n# multicast address to destination of respondd\n# (optional - without definition used default ff05::2:1001)\n# Very old gluon uses \"ff02::2:1001\" as multicast, newer use ff05::2:1001. If you have old and new\n# gluon nodes on the same network, create a separate \"respondd.interfaces\" section for each mutlicast address.\n#multicast_address = \"ff02::2:1001\" (11)\n# define a port to listen\n# if not set or set to 0 the kernel will use a random free port at its own\n#port = 10001 (12)\n
Enable request and collection of data per respondd requests
Delay startup until a multiple of the period since zero time
How often send request per respondd.
It will send UDP packets with multicast address ff05::2:1001
and port 1001
. If a node does not answer after the half time, it will request with the last know address under the port 1001
.
If you have custom respondd fields, you can ask Yanic to also collect these.
It is possible to have multiple custom fields, just add this group again with new parameters (see toml [[array of table]]).
Info
This does not automatically include these fields in the output. The meshviewer-ffrgb output module will include them under \"custom_fields\", but other modules may simply ignore them.
Tables of sites to save stats for (not exists for global only). Here is the site ffhb.
Example
[respondd.sites.ffhb]\ndomains = [\"city\"]\n
list of domains on this site to save stats for (empty for global only)
Interface that has an ip address in your mesh network.
It is possible to have multiple interfaces, just add this group again with new parameters (see toml [[array of table]]).
name of interface on which this collector is running.
ip address is the own address which is used for sending. If not set or set with empty string it will take an address of ifname. (It prefers the link local address, so at babel mesh-network it should be configurated)
Disable sending multicast respondd request. For receiving only respondd packages e.g. database respondd.
Multicast address to destination of respondd. If not set or set with empty string it will take the batman default multicast address ff05::2:1001
(Needed to set for legacy ff02::2:1001
)
Define a port to listen and send the respondd packages. If not set or set to 0 the kernel will use a random free port at its own.
Yanic has a little build-in webserver, which statically serves a directory. This is useful for testing purposes or for a little standalone installation.
# A little build-in webserver, which statically serves a directory.\n# This is useful for testing purposes or for a little standalone installation.\n[webserver]\nenable = false # (1)\nbind = \"127.0.0.1:8080\" # (2)\nwebroot = \"/var/www/html/meshviewer\" # (3)\n
[nodes]\n# Cache file\n# a json file to cache all data collected directly from respondd\nstate_path = \"/var/lib/yanic/state.json\" # (1)\n# prune data in RAM, cache-file and output json files (i.e. nodes.json)\n# that were inactive for longer than\nprune_after = \"7d\" # (2)\n# Export nodes and graph periodically\nsave_interval = \"5s\" # (3)\n# Set node to offline if not seen within this period\noffline_after = \"10m\" # (4)\n
This example block shows all option which is useable for every following output type. Every output type has his own configuration under nodes.output
. It is possible to have multiple output for one type of output, just add this group again with new parameters (see toml [[array of table]]).
[[nodes.output.example]]\nenable = true # (1)\n\n[nodes.output.example.filter] # (2)\nno_owner = true # if it is not set, it will publish contact information of other persons (3)\nblocklist = [\"00112233445566\", \"1337f0badead\"] # (4)\nsites = [\"ffhb\"] # (5)\ndomain_as_site = true # (6)\ndomain_append_site = true # (7)\nhas_location = true # (8)\n\n[nodes.output.example.filter.in_area] # (9)\nlatitude_min = 34.30\nlatitude_max = 71.85\nlongitude_min = -24.96\nlongitude_max = 39.72\n
Set to false, if you want the json files to contain the owner information
Warning
if it is not set, it will publish contact information of other persons.
List of nodeids of nodes that should be filtered out, so they won't appear in output
List of site_codes of nodes that should be included in output
Replace the site_code
with the domain_code
in this output.
e.g. site_code='ffhb',domain_code='city'
becomes site_code='city', domain_code=''
Append on the site_code
the domain_code
with a .
in this output.
e.g. site_code='ffhb',domain_code='city'
becomes site_code='ffhb.city', domain_code=''
set has_location to true if you want to include only nodes that have geo-coordinates set
(setting this to false has no sensible effect, unless you'd want to hide nodes that have coordinates)
nodes outside this area are not shown on the map but are still listed as a node without coordinates
The geojson output produces a geojson file which contains the location data of all monitored nodes to be used to visualize the location of the nodes. It is optimized to be used with UMap but should work with other tools as well.
Here is a public demo provided by Freifunk Muenchen: http://u.osmfr.org/m/328494/
[[nodes.output.geojson]]\nenable = true\npath = \"/var/www/html/meshviewer/data/nodes.geojson\" # (1)\n[nodes.output.geojson.filter]\nno_owner = false\n
The new json file format for the meshviewer developed in Regensburg.
[[nodes.output.meshviewer-ffrgb]]\nenable = true\npath = \"/var/www/html/meshviewer/data/meshviewer.json\" # (1)\n# like on every output, here some filters, for example using this block:\n[nodes.output.meshviewer-ffrgb.filter]\nno_owner = false\nblocklist = [\"00112233445566\", \"1337f0badead\"]\n\n[nodes.output.meshviewer-ffrgb.filter.in_area]\nlatitude_min = 34.30\nlatitude_max = 71.85\nlongitude_min = -24.96\nlongitude_max = 39.72\n
[[nodes.output.meshviewer]]\nenable = false\nversion = 2 # (1)\nnodes_path = \"/var/www/html/meshviewer/data/nodes.json\" # (3)\ngraph_path = \"/var/www/html/meshviewer/data/graph.json\" # (4)\n[nodes.output.meshviewer.filter]\nno_owner = false\n
The structure version of the output which should be generated (i.e. nodes.json)
1
is accepted by the legacy meshviewer (which is the master branch)2
is accepted by the new version of meshviewer (which are in legacy develop branch or newer)nodes_version
)The nodelist output is a minimal output with current state of collected data. Should be preferred to use it on the ffapi for the freifunk-karte.de
[[nodes.output.nodelist]]\nenable = false\npath = \"/var/www/html/meshviewer/data/nodelist.json\" # (1)\n[nodes.output.nodelist.filter]\nno_owner = false\n
The Prometheus Service Discovery (SD) output is a output with the list of addresses of the nodes to use them in later exporter by prometheus. For usage in Prometheus read there Documentation Use file-based service discovery to discover scrape targets.
[[nodes.output.prometheus-sd]]\nenable = false\npath = \"/var/www/html/meshviewer/data/prometheus-sd.json\" # (1)\n# ip = lates recieved ip, node_id = node id from host\ntarget_address = \"ip\" # (2)\n\n# Labels of the data (optional)\n[nodes.output.prometheus-sd.labels] # (3)\nlabelname1 = \"labelvalue 1\"\n# some useful e.g.:\nhosts = \"ffhb\"\nservice = \"yanic\"\n
In the prometheus-sd.json the usage of which information of the node as targets (address).
Use the node_id
as value, to put the Node ID into the target list as address.
Use the ip
as value to put the last IP address into the target list from where the respondd message is recieved (maybe a link-local address).
Default value is ip
.
You could optional set manuelle labels with inserting into a prometheus-sd.json. Useful if you want to identify the yanic instance when you use multiple own on the same prometheus database (e.g. multisites).
This output takes the respondd response as sent by the node and includes it in a JSON document.
[[nodes.output.raw]]\nenable = false\npath = \"/var/www/html/meshviewer/data/raw.json\" # (1)\n[nodes.output.raw.filter]\nno_owner = false\n
This output takes the respondd response as sent by the node and inserts it into a line-separated JSON document (JSONL). In this format, each line can be interpreted as a separate JSON element, which is useful for json streaming. The first line is a json object containing the timestamp and version of the file. This is followed by a line for each node, each containing a json object.
[[nodes.output.raw-jsonl]]\nenable = false\npath = \"/var/www/html/meshviewer/data/raw.jsonl\" # (1)\n[nodes.output.raw-jsonl.filter]\nno_owner = false\n
The database organize all database types. For all database types the is a internal job, which reset data for nodes (global statistics are still stored). (We have for privacy policy to store node data for maximum seven days.)
[database]\ndelete_after = \"7d\" # (1)\ndelete_interval = \"1h\" # (2)\n
This example block shows all option which is useable for every following database type. Every database type has his own configuration under database.connection
. It is possible to have multiple connections for one type of database, just add this group again with new parameters (see toml [[array of table]]).
[[database.connection.example]]\nenable = true # (1)\n
Save collected data to InfluxDB. There are would be the following measurements: - node: store node specific data i.e. clients memory, airtime - link: store link tq between two interfaces of two different nodes - global: store global data, i.e. count of clients and nodes - firmware: store the count of nodes tagged with firmware - model: store the count of nodes tagged with hardware model - autoupdater: store the count of autoupdate branch
[[database.connection.influxdb]]\nenable = false\naddress = \"http://localhost:8086\" # (1)\ndatabase = \"ffhb\" # (2)\nusername = \"\" # (3)\npassword = \"\" # (4)\n# insecure_skip_verify = true (5)\n\n[database.connection.influxdb.tags] # (6)\ntagname1 = \"tagvalue 1\"\nsystem = \"productive\"\nsite = \"ffhb\"\n
You could set manuelle tags with inserting into a influxdb.
Useful if you want to identify the yanic instance when you use multiple own on the same influxdb (e.g. multisites).
Warning
Tags used by Yanic would override the tags from this config (e.g. nodeid
, hostname
, owner
, model
, firmware_base
, firmware_release
, frequency11g
, frequency11a
).
Save collected data to InfluxDB2.
There are the following measurements:
Info
A bucket has to be set in buckets and buchet_default otherwise yanic would panic.
Warning
yanic does NOT prune node's data (so please set up data retention in InfluxDB2 setup).
We highly recommend to setup e.g. Data retention in your InfluxDB2 server per measurements.
[[database.connection.influxdb2]]\nenable = false\naddress = \"http://localhost:8086\" # (1)\ntoken = \"\" # (2)\norganization_id = \"\" # (3)\nbucket_default = \"yanic\" # (4)\n\n[database.connection.influxdb2.buckets] # (5)\n#link = \"yanic-temp\"\n#node = \"yanic-temp\"\n#dhcp = \"yanic-temp\"\nglobal = \"yanic-persistent\"\n#firmware = \"yanic-temp\"\n#model = \"yanic-temp\"\n#autoupdater = \"yanic-temp\"\n\n# Tagging of the data (optional)\n[database.connection.influxdb2.tags] # (6)\n# Tags used by Yanic would override the tags from this config\n# nodeid, hostname, owner, model, firmware_base, firmware_release,frequency11g and frequency11a are tags which are already used\n#tagname1 = \"tagvalue 1\"\n# some useful e.g.:\n#system = \"productive\"\n#site = \"ffhb\"\n
Set organization using the InfluxDB2 server.
Bucket in which are the data stored.
Fallback of bucket per measurement, see [database.connection.influxdb2.buckets]
Buckets per measurement.
If no buckets are set, the default bucket bucket_default
is used.
You can set additional tags which are added to all data written into the influxdb.
Useful if you want to identify the yanic instance when you use multiple own on the same influxdb (e.g. multisites).
Warning
Tags used by Yanic would override the tags from this config (e.g. nodeid
, hostname
, owner
, model
, firmware_base
, firmware_release
, frequency11g
, frequency11a
).
Save collected data to a graphite database.
# Graphite settings\n[database.connection.graphite]]\nenable = false\naddress = \"localhost:2003\" # (1)\n# Graphite is replacing every \".\" in the metric name with a slash \"/\" and uses\n# that for the file system hierarchy it generates. it is recommended to at least\n# move the metrics out of the root namespace (that would be the empty prefix).\n# If you only intend to run one community and only freifunk on your graphite node\n# then the prefix can be set to anything (including the empty string) since you\n# probably wont care much about \"polluting\" the namespace.\nprefix = \"freifunk\" # (2)\n
Forward collected respondd package to a address (e.g. to another respondd collector like a central yanic instance or hopglass)
# respondd (yanic)\n# forward collected respondd package to a address\n# (e.g. to another respondd collector like a central yanic instance or hopglass)\n[[database.connection.respondd]]\nenable = false\n# type of network to create a connection\ntype = \"udp6\" # (1)\n# destination address to connect/send respondd package\naddress = \"stats.bremen.freifunk.net:11001\" # (2)\n
Type of network to create a connection.
Known networks are \"tcp\", \"tcp4\" (IPv4-only), \"tcp6\" (IPv6-only), \"udp\", \"udp4\" (IPv4-only), \"udp6\" (IPv6-only), \"ip\", \"ip4\" (IPv4-only), \"ip6\" (IPv6-only), \"unix\", \"unixgram\" and \"unixpacket\".
Destination address to connect/send respondd package.
This database type is just for, debugging without a real database connection. A example for other developers for new database types.
# Logging\n[[database.connection.logging]]\nenable = false\npath = \"/var/log/yanic.log\" # (1)\n
cd /usr/local/\nwget https://dl.google.com/go/go1.13.1.linux-amd64.tar.gz -O go-release-linux-amd64.tar.gz\ntar xvf go-release-linux-amd64.tar.gz\nrm go-release-linux-amd64.tar.gz\n
"},{"location":"docs/install.html#configure-go","title":"Configure go","text":"Add these lines in your root shell startup file (e.g. /root/.bashrc
):
export GOPATH=/opt/go\nexport PATH=$PATH:/usr/local/go/bin:$GOPATH/bin\n
"},{"location":"docs/install.html#yanic","title":"Yanic","text":""},{"location":"docs/install.html#compile","title":"Compile","text":"As root:
go install github.com/FreifunkBremen/yanic@latest\n
or to install a different checkout for example for development run:
git clone https://github.com/FreifunkBremen/yanic\ncd yanic\ngo install .\n
"},{"location":"docs/install.html#install_1","title":"Install","text":"cp /opt/go/src/github.com/FreifunkBremen/yanic/contrib/init/linux-systemd/yanic.service /lib/systemd/system/yanic.service\nsystemctl daemon-reload\n
Before start, you should configure yanic by the file /etc/yanic.conf
:
systemctl start yanic\n
Enable to start on boot:
systemctl enable yanic\n
"},{"location":"docs/install.html#update","title":"Update","text":"For an update just stop yanic and then call the same go
command again (again as root):
systemctl stop yanic\ngo get -v -u github.com/FreifunkBremen/yanic\n
Then update the config file, for example look at the diff with the new example: diff /opt/go/src/github.com/FreifunkBremen/yanic/config_example.toml /etc/yanic.conf\n
"},{"location":"docs/quick_conf.html","title":"Quick Configuration","text":"cp /opt/go/src/github.com/FreifunkBremen/yanic/config_example.toml /etc/yanic.conf\n
"},{"location":"docs/quick_conf.html#configuration","title":"Configuration","text":"For an easy startup you only need to edit the [[respondd.interfaces]]
in section [respondd]
in file /etc/yanic.conf
.
Then create the following files and folders:
adduser --system yanic --home /var/lib/yanic\nmkdir -p /var/lib/yanic\nmkdir -p /var/www/html/meshviewer/data\ntouch /var/log/yanic.log\nchown yanic /var/log/yanic.log /var/lib/yanic /var/www/html/meshviewer/data\n
"},{"location":"docs/quick_conf.html#standalone","title":"Standalone","text":"If you like to run a standalone meshviewer, just set enable
in section [webserver]
to true
.
set dataPath
in config.json
to /data/
and make the build
directory accessible under /var/www/html/meshviewer
.
The meshviewer needs the output files like nodes_path
and graph_path
inside the same directory as the dataPath
. Change the path in the section [meshviewer]
accordingly.
Yanic provides several commands:
import
query
serve
Warning, just tested with olddata.rrd from Freifunk Bremen generated by detailed-rrds branch of ffmap-backend
Usage:\n yanic import <file.rrd> [flags]\n\nExamples:\n yanic import --config /etc/yanic.toml olddata.rrd\n\nFlags:\n -c, --config string Path to configuration file (default \"config.toml\")\n -h, --help help for import\n
"},{"location":"docs/usage.html#firstseen","title":"Firstseen","text":"To import firstseen values there is a little script in contrib:
/opt/go/src/github.com/FreifunkBremen/yanic/contrib/yanic-import-timestamp -n path/to/nodes_old.json -s state.json /var/lib/yanic/state.json\n
On a productive system @ once:
systemctl stop yanic; cp /var/lib/yanic/state.json /var/lib/yanic/state.bak; /opt/go/src/github.com/FreifunkBremen/yanic/contrib/yanic-import-timestamp -n path/to/nodes_old.json -s /var/lib/yanic/state.json; systemctl start yanic;\n
"},{"location":"docs/usage.html#serve","title":"Serve","text":"runs yanic in collector-modus to genereate files (e.g. for meshviewer) and save values in databases
from shell
Usage:\n yanic serve [flags]\n\nExamples:\n yanic serve --config /etc/yanic.toml\n\nFlags:\n -c, --config string Path to configuration file (default \"config.toml\")\n -h, --help help for serve\n
or run as daemon
"},{"location":"docs/usage.html#query","title":"Query","text":"Send a single request and show response like gluon-neighbour-info
on gluon.
e.g. to check the right interface
Usage:\n yanic query <interface> <destination> [flags]\n\nExamples:\n yanic query wlan0 \"fe80::eade:27ff:dead:beef\"\n\nFlags:\n -h, --help help for query\n --wait int Seconds to wait for a response (default 1)\n
"}]}