How to protect an instance¶
Tip
To protect your instance a installation of filtron (as described here) is no
longer needed, alternatively activate the Limiter Plugin in your
settings.yml. Note that the Limiter Plugin requires a Redis database.
SearXNG depends on external search services. To avoid the abuse of these services it is advised to limit the number of requests processed by SearXNG.
An application firewall, filtron solves exactly this problem. Filtron is just a middleware between your web server (nginx, apache, …) and searx, we describe such infrastructures in chapter: Architecture.
filtron & go¶
Filtron needs Go installed. If Go is preinstalled, filtron is simply
installed by go get package management (see filtron README). If you use
filtron as middleware, a more isolated setup is recommended. To simplify such
an installation and the maintenance of, use our script utils/filtron.sh.
Sample configuration of filtron¶
An example configuration can be find below. This configuration limits the access of:
scripts or applications (roboagent limit)
webcrawlers (botlimit)
IPs which send too many requests (IP limit)
too many json, csv, etc. requests (rss/json limit)
the same UserAgent of if too many requests (useragent limit)
[
{
"name": "search request",
"filters": [
"Param:q",
"Path=^(/|/search)$"
],
"interval": "<time-interval-in-sec (int)>",
"limit": "<max-request-number-in-interval (int)>",
"subrules": [
{
"name": "missing Accept-Language",
"filters": ["!Header:Accept-Language"],
"limit": "<max-request-number-in-interval (int)>",
"stop": true,
"actions": [
{"name":"log"},
{"name": "block",
"params": {"message": "Rate limit exceeded"}}
]
},
{
"name": "suspiciously Connection=close header",
"filters": ["Header:Connection=close"],
"limit": "<max-request-number-in-interval (int)>",
"stop": true,
"actions": [
{"name":"log"},
{"name": "block",
"params": {"message": "Rate limit exceeded"}}
]
},
{
"name": "IP limit",
"interval": "<time-interval-in-sec (int)>",
"limit": "<max-request-number-in-interval (int)>",
"stop": true,
"aggregations": [
"Header:X-Forwarded-For"
],
"actions": [
{ "name": "log"},
{ "name": "block",
"params": {
"message": "Rate limit exceeded"
}
}
]
},
{
"name": "rss/json limit",
"filters": [
"Param:format=(csv|json|rss)"
],
"interval": "<time-interval-in-sec (int)>",
"limit": "<max-request-number-in-interval (int)>",
"stop": true,
"actions": [
{ "name": "log"},
{ "name": "block",
"params": {
"message": "Rate limit exceeded"
}
}
]
},
{
"name": "useragent limit",
"interval": "<time-interval-in-sec (int)>",
"limit": "<max-request-number-in-interval (int)>",
"aggregations": [
"Header:User-Agent"
],
"actions": [
{ "name": "log"},
{ "name": "block",
"params": {
"message": "Rate limit exceeded"
}
}
]
}
]
}
]
Route request through filtron¶
Filtron can be started using the following command:
$ filtron -rules rules.json
It listens on 127.0.0.1:4004 and forwards filtered requests to
127.0.0.1:8888 by default.
Use it along with nginx with the following example configuration.
# https://example.org/searx
location /searx {
proxy_pass http://127.0.0.1:4004/;
proxy_set_header Host $host;
proxy_set_header Connection $http_connection;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Scheme $scheme;
proxy_set_header X-Script-Name /searx;
}
location /searx/static {
/usr/local/searx/searx-src/searx/static;
}
Requests are coming from port 4004 going through filtron and then forwarded to port 8888 where a SearXNG is being run. For a complete setup see: A nginx SearXNG site.
